Andy3
(Andy)
October 26, 2015, 8:06am
1
Hi expert,
During use ansible to update package on all the hosts by yum module, ansible will hung at the end, below is the command I run:
ansible devstack -m yum -a “name=wget state=latest” -vvvv
devstack is a group which contain about 70 IP.
Any method I can try to debug this issue?
-BR,
Andy
Brian_Coca
(Brian Coca)
October 26, 2015, 6:08pm
2
check on the target machine on which it is stuck, is yum itself stuck?
are there resource issues? what is the ansible process there stuck on
(strace)?
Also are you using 1.9.4 or a checkout/beta of 2.0 and do you have the
rhn-plugin enabled? I just closed a ticket for 1.9.x where the
culprit was repoquery. In 2.0 we make much less frequent use of
repoquery so it's possible that the 2.0 module has "fixed" this. If
it is also repoquery here, you might have to pursue a bugfix with the
repoquery authors.
-Toshio
Andy3
(Andy)
October 28, 2015, 9:58am
4
Yes, you are right, ansible stuck on a target two machines, and I attach to the process on one of the machine but no useful information I can get:
`
[root@meil-juno ~]# ps axu | grep ansible
root 31477 0.0 0.1 259388 22940 ? S 09:19 0:00 /usr/bin/python -tt /root/.ansible/tmp/ansible-tmp-1446023983.89-103656620169766/yum
root 31559 0.0 0.0 113116 1200 pts/1 Ss+ 09:22 0:00 /bin/sh -c LANG=C LC_CTYPE=C /usr/bin/python -tt /root/.ansible/tmp/ansible-tmp-1446024152.45-82112154264775/yum; rm -rf /root/.ansible/tmp/ansible-tmp-1446024152.45-82112154264775/ >/dev/null 2>&1
root 31570 0.0 0.1 259388 22936 pts/1 S+ 09:22 0:00 /usr/bin/python -tt /root/.ansible/tmp/ansible-tmp-1446024152.45-82112154264775/yum
root 31913 0.0 0.0 112640 968 pts/2 S+ 09:51 0:00 grep --color=auto ansible[root@meil-juno ~]# strace -p 31477
Process 31477 attached
fcntl(4, F_SETLKW, {type=F_WRLCK, whence=SEEK_SET, start=0, len=0}
^CProcess 31477 detached
<detached …>
[root@meil-juno ~]# strace -p 31570
Process 31570 attached
fcntl(4, F_SETLKW, {type=F_WRLCK, whence=SEEK_SET, start=0, len=0}
`
No more output there, so do you have any idea to debug this problem?
BTW, my ansible version is: 1.9.4
-BR,
Andy
在 2015年10月27日星期二 UTC+8上午2:08:38,Brian Coca写道:
Brian_Coca
(Brian Coca)
October 28, 2015, 12:24pm
5
the processes are stuck waiting for a lock, the package database is
the most probable culprit. check if any other process is running yum
or rpm.
You can check te eexact file locked using `lsof -p <pid>` and check
for the 4 descriptor.