AWX syncronize built in stopped working. No error message just stalls out

Archives AWX Project
1

After pulling Gallaxy content the synchronize built just stops and hangs at the login to the remote host. Worked perfectly previously, I have blown away AWx and reinstalled it completely and still the same results.

Using synchronize with delegated host to copy files from repos.

Any thoughts on what would cause this.

sshpass -d10 ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o User="*-admin" -o ConnectTimeout=10 -o ControlPath=“/runner/cp/244a2ad7c3” -tt **********02 /bin/sh -c '/usr/bin/python3 /home/-admin/.ansible/tmp/ansible-tmp-1675369700.6906097-45-62275435632762/AnsiballZ_synchronize.py && sleep 0’

1000 3850516 3850515 0 14:28 pts/2 00:00:00 ssh -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o User=“******-admin” -o ConnectTimeout=10 -o ControlPath=“/runner/cp/244a2ad7c3” -tt ******02 /bin/sh -c '/usr/bin/python3 /home/-admin/.ansible/tmp/ansible-tmp-1675369700.6906097-45-62275435632762/AnsiballZ_synchronize.py && sleep 0’

*****-a+ 3850554 3850548 0 14:28 pts/4 00:00:00 /usr/bin/rsync --delay-updates -F --compress --checksum --archive --no-recursive --rsh=/usr/bin/ssh -S none -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null --rsync-path=sudo -u ******-admin rsync --out-format=<>%i %n%L /software/packages/cisco/current/ ******-admin@***********01:/tmp/software/cisco

All processes are waiting, below is the last created by the ansible. Killing this one kills all.

******-a+ 3850555 3850554 0 14:28 pts/4 00:00:00 /usr/bin/ssh -S none -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -l ******-admin **********01 sudo -u *****-admin rsync --server -logDtpcze.LsfxC --log-format=%i --delay-updates . /tmp/software/cisco
root 3851196 3847802 0 14:31 pts/3 00:00:00 grep --color=auto ssh

2

Hi!

do you see ansible processes (python processes that called the module) hanging on the remote node?

do your playbooks run fine outside of AWX?

“Worked perfectly previously”
what do you mean exactly? Like a previous version of AWX?

AWX Team

3

Hi

Processes don't even show up on the remote node...strangely.

Yes they run fine using ansible-playbook, I think this is related to an issue that others had with issues after upgrading to latest image. This was working fine until I installed a watch agent from Galaxy.

After making some changes I see the "you must install sshpass..." error.

I am currently rolling back to 21.11.0

4

Rolling back didn’t work same issue. Syncronize jobs hang, these appear in the process list as ssh processes, AWX just running but not completing. Checking to see if docker vbridge could be blocking.

5

Something has definitely change to cause this to stop working. The difference between running from AWX is substantial as oppose to ansible-playbook. BTW the same play runs fine with ansible-playbook. All playbooks work fine that are not using synchronize that I can tell or that I have tested in AWX. All work when running from the CLI

6

Are you succeeding with CLI on the same host that AWX is failing?

Walter

7

Also, running the playbook is a connection from control host to target machine. The git sync is a connection between the control host and the git repo host.

Walter

8

I am succeeding on the same host.

9

At one point I thought maybe this is a system issue with SSH but since the CLI playbook runs fine. I have changed my thinking on that. I can see the system processes spin up by listing the processes. The last playbook process in the chain just sits and spins forever.

10

After making some changes I see the “you must install sshpass…” error.

Are you using ssh keys or ssh username/password?

Walter