# Strange problem running Ansible against a CentOS 7 box

**URL:** <https://forum.ansible.com/t/strange-problem-running-ansible-against-a-centos-7-box/16115>\
**Category:** Ansible Project\
**Tags:** ubuntu\
**Created:** [September 4, 2014, 5:04pm UTC](https://forum.ansible.com/t/strange-problem-running-ansible-against-a-centos-7-box/16115 "2014-09-04T17:04:30Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![Will\_Dennis](https://avatars.discourse-cdn.com/v4/letter/w/919ad9/32.png) [@Will\_Dennis](https://forum.ansible.com/u/Will_Dennis)\
**Post date:** [September 4, 2014, 5:04pm UTC](https://forum.ansible.com/t/strange-problem-running-ansible-against-a-centos-7-box/16115/1 "2014-09-04T17:04:30Z")

</div>

Hi all,

I just installed CentOS 7 on a new machine, and ‘yum update’-ed it to pick up the latest packages. Here’s the output of ‘uname -a’ and ‘/etc/redhat-release’:

[root@problem-svr ~]# uname -a  
Linux [problem-svr.mycompany.com](http://problem-svr.mycompany.com) 3.10.0-123.6.3.el7.x86\_64 #1 SMP Wed Aug 6 21:12:36 UTC 2014 x86\_64 x86\_64 x86\_64 GNU/Linux  
[root@problem-svr ~]# cat /etc/redhat-release  
CentOS Linux release 7.0.1406 (Core)

When I try to do anything with Ansible (v1.7.1 running on Ubuntu 12.04.5) against this box, it just hangs (even ‘-m ping’) When I throw the ‘-vvvv’ on the run, here’s what I see:

will@wdennis-p390:~/ansible-stuff$ ansible -vvvv problem-svr -u root -k -i test -m setup  
SSH password:  
 ESTABLISH CONNECTION FOR USER: root  
 REMOTE\_MODULE setup  
 EXEC [‘sshpass’, ‘-d6’, ‘ssh’, ‘-C’, ‘-tt’, ‘-vvv’, ‘-o’, ‘ControlMaster=auto’, ‘-o’, ‘ControlPersist=60s’, ‘-o’, ‘ControlPath=/home/will/.ansible/cp/ansible-ssh-%h-%p-%r’, ‘-o’, ‘Port=22’, ‘-o’, ‘GSSAPIAuthentication=no’, ‘-o’, ‘PubkeyAuthentication=no’, ‘-o’, ‘User=root’, ‘-o’, ‘ConnectTimeout=10’, ‘problem-svr-new’, “/bin/sh -c ‘mkdir -p $HOME/.ansible/tmp/ansible-tmp-1409846776.31-88290040276656 && echo $HOME/.ansible/tmp/ansible-tmp-1409846776.31-88290040276656’”]

I do see a SSH session initiated on the host:

[root@problem-svr ~]# ss -4 -t  
State Recv-Q Send-Q Local Address:Port Peer Address:Port  
ESTAB 0 208 192.168.180.22:ssh 192.168.180.50:63172  
[root@problem-svr ~]# ss -4 -t  
State Recv-Q Send-Q Local Address:Port Peer Address:Port  
ESTAB 0 0 192.168.180.22:ssh 192.168.180.53:42717 \<— Ansible session  
ESTAB 0 0 192.168.180.22:ssh 192.168.180.50:63172

But then, the session just times out and finally drops:

State Recv-Q Send-Q Local Address:Port Peer Address:Port  
FIN-WAIT-2 0 0 192.168.180.22:ssh 192.168.180.53:42717  
ESTAB 0 208 192.168.180.22:ssh 192.168.180.50:63172  
[root@problem-svr ~]# ss -4 -t  
State Recv-Q Send-Q Local Address:Port Peer Address:Port  
ESTAB 0 208 192.168.180.22:ssh 192.168.180.50:63172

Meanwhile, the Ansible process on the control machine keeps trying (i.e., does not die when the session ends) and eventually, I kill it with a Ctrl-C.

I did already try setting SELinux to “disabled” on the CentOS 7 box, and turning off the ‘firewalld’ service (does not seem to make a difference.)

I do have another CentOS 7 box that I can successfully run Ansible against, so I think it’s just something strange on the target CentOS 7 box… How can I further debug this?

Thanks,  
Will

---

<div class="post-metadata">

**Author:** ![Dick\_Davies](https://avatars.discourse-cdn.com/v4/letter/d/b9e5f3/32.png) [@Dick\_Davies](https://forum.ansible.com/u/Dick_Davies)\
**Post date:** [September 4, 2014, 5:40pm UTC](https://forum.ansible.com/t/strange-problem-running-ansible-against-a-centos-7-box/16115/2 "2014-09-04T17:40:49Z")

</div>

Huh weird - I've started porting some of our centos6 play books over  
to centos7 and didn't have  
any trouble (OSX client, pure ssh transport) but that was using SSH  
pubkey auth.

Maybe there's something up with the way centos7 does password auth?

I'm guessing you can ssh straight in as the ansible user with the same pass etc?  
(If not, fix that first 🙂 )

If so I'd check /var/log/secure and see if there are any differences  
in how sshd is  
seeing the sessions of the ansible connection vs. your vanilla ssh client.

---

<div class="post-metadata">

**Author:** ![Will\_Dennis](https://avatars.discourse-cdn.com/v4/letter/w/919ad9/32.png) [@Will\_Dennis](https://forum.ansible.com/u/Will_Dennis)\
**Post date:** [September 4, 2014, 6:12pm UTC](https://forum.ansible.com/t/strange-problem-running-ansible-against-a-centos-7-box/16115/3 "2014-09-04T18:12:30Z")

</div>

I can indeed SSH straight in (using ‘root’ with password.)

I made sure “PermitRootLogin” was explicitly set to ‘yes’ in sshd\_config, restarted sshd, and tried again. The Ansible command still hangs, and no messages in /var/log/secure, other than when I kill the Ansible process, it reports “Connection closed”:

Sep 4 14:06:34 problem-svr sshd[1457]: Received signal 15; terminating.  
Sep 4 14:06:34 problem-svr sshd[17358]: Server listening on 0.0.0.0 port 22.  
Sep 4 14:06:34 problem-svr sshd[17358]: Server listening on :: port 22.  
Sep 4 14:07:16 problem-svr sshd[17360]: Connection closed by 192.168.180.53 [preauth]

Very strange & frustrating…

---

<div class="post-metadata">

**Author:** ![John\_Favorite](https://avatars.discourse-cdn.com/v4/letter/j/ba9def/32.png) [@John\_Favorite](https://forum.ansible.com/u/John_Favorite)\
**Post date:** [September 4, 2014, 6:22pm UTC](https://forum.ansible.com/t/strange-problem-running-ansible-against-a-centos-7-box/16115/4 "2014-09-04T18:22:28Z")

</div>

Are there any differences between the centos 7 box that works and the new one? I would be looking at a recent package updates.

---

<div class="post-metadata">

**Author:** ![mhael](https://avatars.discourse-cdn.com/v4/letter/m/919ad9/32.png) [@mhael](https://forum.ansible.com/u/mhael)\
**Post date:** [September 4, 2014, 7:07pm UTC](https://forum.ansible.com/t/strange-problem-running-ansible-against-a-centos-7-box/16115/5 "2014-09-04T19:07:19Z")

</div>

Just a wild guess - can you try running ansible-playbook with ANSIBLE\_HOST\_KEY\_CHECKING=False ?

---

<div class="post-metadata">

**Author:** ![Will\_Dennis](https://avatars.discourse-cdn.com/v4/letter/w/919ad9/32.png) [@Will\_Dennis](https://forum.ansible.com/u/Will_Dennis)\
**Post date:** [September 4, 2014, 7:31pm UTC](https://forum.ansible.com/t/strange-problem-running-ansible-against-a-centos-7-box/16115/6 "2014-09-04T19:31:53Z")

</div>

Wild guess was CORRECT - the runs work now.

So, what could have changed on this box that “export ANSIBLE\_HOST\_KEY\_CHECKING=False” would have fixed? (not a SSH guru here… pls educate me)

Thanks,  
Will

---

<div class="post-metadata">

**Author:** ![John\_Favorite](https://avatars.discourse-cdn.com/v4/letter/j/ba9def/32.png) [@John\_Favorite](https://forum.ansible.com/u/John_Favorite)\
**Post date:** [September 4, 2014, 7:37pm UTC](https://forum.ansible.com/t/strange-problem-running-ansible-against-a-centos-7-box/16115/7 "2014-09-04T19:37:46Z")

</div>

Here’s what I found: [http://docs.ansible.com/intro\_getting\_started.html#host-key-checking](http://docs.ansible.com/intro_getting_started.html#host-key-checking)

---

<div class="post-metadata">

**Author:** ![mhael](https://avatars.discourse-cdn.com/v4/letter/m/919ad9/32.png) [@mhael](https://forum.ansible.com/u/mhael)\
**Post date:** [September 4, 2014, 7:57pm UTC](https://forum.ansible.com/t/strange-problem-running-ansible-against-a-centos-7-box/16115/8 "2014-09-04T19:57:52Z")

</div>

CentOS 7 box is probably fine and this is local issue with stored SSH keys. When you connecting via ssh directly - does it asks you anything (f.e. about mismatching keys) ?  
If yes - it should provide the offending line # in .ssh/known\_hosts. Try to remove this line, then ssh directly to the host to reacquire host key, and then try to run ansible-playbook.

You can also run these commands instead of editing known\_hosts file manually (as the same user you run ansible from):  
ssh-keygen -f ‘~/.ssh/known\_hosts’ -R \<centos7\_box\_ip\>  
ssh-keygen -f ‘~/.ssh/known\_hosts’ -R \<centos7\_box\_hostname\>

---

<div class="post-metadata">

**Author:** ![Will\_Dennis](https://avatars.discourse-cdn.com/v4/letter/w/919ad9/32.png) [@Will\_Dennis](https://forum.ansible.com/u/Will_Dennis)\
**Post date:** [September 4, 2014, 11:52pm UTC](https://forum.ansible.com/t/strange-problem-running-ansible-against-a-centos-7-box/16115/9 "2014-09-04T23:52:54Z")

</div>

Yes, I had to reinstall this machine, and did fix the hostname entry in known\_hosts, but did not fix the IP address entry. Good to know the cause and the fix, thanks!

Will

---

<div class="post-metadata">

**Author:** ![Michael\_DeHaan1](https://avatars.discourse-cdn.com/v4/letter/m/5e9695/32.png) [@Michael\_DeHaan1](https://forum.ansible.com/u/Michael_DeHaan1)\
**Post date:** [September 5, 2014, 1:38am UTC](https://forum.ansible.com/t/strange-problem-running-ansible-against-a-centos-7-box/16115/10 "2014-09-05T01:38:35Z")

</div>

CentOS 7 is, FWIW, in our QA matrix and we haven’t seen problems here.

It seems like John is suggesting above that host key checking disabling fixes something for you, but I’m also a bit unclear, as the system will prompt you (ansible will ask you questions) when it is turned on.

It seems like you may have not been seeing the prompts?

Can you clarify a bit perhaps?

Thanks!

---

<div class="post-metadata">

**Author:** ![Will\_Dennis](https://avatars.discourse-cdn.com/v4/letter/w/919ad9/32.png) [@Will\_Dennis](https://forum.ansible.com/u/Will_Dennis)\
**Post date:** [September 5, 2014, 2:33am UTC](https://forum.ansible.com/t/strange-problem-running-ansible-against-a-centos-7-box/16115/11 "2014-09-05T02:33:55Z")

</div>

Hi Michael,

Yes, when I initially reinstalled CentOS on this host, and tried SSH-ing in to it from my Ansible workstation, I did get the typical error about the fact that the host key had changed, and SSH refused to connect. So when I edited the ~/.ssh/known\_hosts file, I just took out the line for the hostname key, and not the one for that host’s IP address. I could then SSH to the host successfully, but there _was_ a warning about the IP key, which I ignored because a) I knew what the cause was, and b) it didn’t terminate the SSH session. I didn’t know that Ansible would refuse to complete the SSH connection (looked like a hang to me - no error was thrown during the Ansible run, even with -vvvv) if one of the two known\_hosts keys were wrong. Of course, turning Ansible’s host key checking off “fixed” the problem, whereupon running Ansible against this specific host then worked.

So in the end, a win for me - learned something about SSH and Ansible today 🙂

Thanks,

Will