Unable to start the glusterd while trying the upgrade test

[rkothiya]

While trying to write an upgrade test, I was unable to start the glusterd

[root@node-1 ~]# systemctl status glusterd
● glusterd.service - GlusterFS, a clustered file-system server
   Loaded: loaded (/usr/lib/systemd/system/glusterd.service; enabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Wed 2021-03-10 09:28:05 UTC; 3h 12min ago
     Docs: man:glusterd(8)
  Process: 24046 ExecStart=/usr/sbin/glusterd -p /var/run/glusterd.pid --log-level $LOG_LEVEL $GLUSTERD_OPTIONS (code=exited, status=1/FAILURE)

Mar 10 09:28:05 node-1 systemd[1]: Starting GlusterFS, a clustered file-system server...
Mar 10 09:28:05 node-1 systemd[1]: glusterd.service: control process exited, code=exited status=1
Mar 10 09:28:05 node-1 systemd[1]: Failed to start GlusterFS, a clustered file-system server.
Mar 10 09:28:05 node-1 systemd[1]: Unit glusterd.service entered failed state.
Mar 10 09:28:05 node-1 systemd[1]: glusterd.service failed.
[root@node-1 ~]# 

I was seeing the following error in the glusterd logs :

[2021-03-10 09:28:04.821340 +0000] W [mgmt-pmap.c:132:rpc_clnt_mgmt_pmap_signout] 0-glusterfs: failed to create XDR payload
[2021-03-10 09:28:05.020540 +0000] I [MSGID: 100030] [glusterfsd.c:2699:main] 0-/usr/sbin/glusterd: Started running version [{arg=/usr/sbin/glusterd}, {version=20210309.dc9bab7}, {cmdlinestr=/usr/sbin/glusterd -p /var/run/glusterd.pid --log-level INFO}] 
[2021-03-10 09:28:05.021667 +0000] I [glusterfsd.c:2434:daemonize] 0-glusterfs: Pid of current running process is 24047
[2021-03-10 09:28:05.024569 +0000] I [MSGID: 106478] [glusterd.c:1472:init] 0-management: Maximum allowed open file descriptors set to 65536 
[2021-03-10 09:28:05.024672 +0000] I [MSGID: 106479] [glusterd.c:1548:init] 0-management: Using /var/lib/glusterd as working directory 
[2021-03-10 09:28:05.024690 +0000] I [MSGID: 106479] [glusterd.c:1554:init] 0-management: Using /var/run/gluster as pid file working directory 
[2021-03-10 09:28:05.041592 +0000] I [socket.c:983:__socket_server_bind] 0-socket.management: process started listening on port (24007)
[2021-03-10 09:28:05.043605 +0000] I [socket.c:929:__socket_server_bind] 0-socket.management: closing (AF_UNIX) reuse check socket 12
[2021-03-10 09:28:05.044143 +0000] I [MSGID: 106059] [glusterd.c:1939:init] 0-management: max-port override: 60999 
[2021-03-10 09:28:05.045781 +0000] E [MSGID: 106229] [glusterd.c:513:glusterd_check_gsync_present] 0-glusterd: geo-replication module not working as desired 
[2021-03-10 09:28:05.045947 +0000] E [MSGID: 101019] [xlator.c:641:xlator_init] 0-management: Initialization of volume failed. review your volfile again. [{name=management}] 
[2021-03-10 09:28:05.045965 +0000] E [MSGID: 101066] [graph.c:425:glusterfs_graph_init] 0-management: initializing translator failed 
[2021-03-10 09:28:05.045986 +0000] E [MSGID: 101176] [graph.c:777:glusterfs_graph_activate] 0-graph: init failed 
[2021-03-10 09:28:05.046302 +0000] W [glusterfsd.c:1445:cleanup_and_exit] (-->/usr/sbin/glusterd(glusterfs_volumes_init+0xaa) [0x560bfc8e17ca] -->/usr/sbin/glusterd(glusterfs_process_volfp+0x236) [0x560bfc8e1706] -->/usr/sbin/glusterd(cleanup_and_exit+0x6b) [0x560bfc8e0a5b] ) 0-: received signum (-1), shutting down 
[2021-03-10 09:28:05.046381 +0000] W [mgmt-pmap.c:132:rpc_clnt_mgmt_pmap_signout] 0-glusterfs: failed to create XDR payload
Teardownclass: TestDfBeforeAndAfterUpgrade_cplex_distributed-dispersed_glusterfs : 09_24_10_03_2021

But Once I installed glusterfs-geo-replication and then tried to restart it did not give any problem.

[mykaul]

Looks like a dependency issue to me. I can probably add it manually to the RPMs deployed. I'm surprised it prevents the run and fails so miserably.

[rkothiya]

I have added the dependency with the above patch.

[mykaul]

How is it not a RPM spec issue?

[rkothiya]

If it was rpm spec issue, I would not have been able to upgrade to the latest build.
I was able to upgrade it but glusterd was not starting. I think there is a dependency which has got introduced in the code.