Ansible SSH known-hosts.

Archives Ansible Project
1

Hi,

Ansible: 1.3.2

I have a question related how ansible is working with know hosts.
I am trying run ansible from jenkins on local VMs managed by vagrant.

When I’ve run my playbook from jenkins - I’ve started having this error:

=== Setup build hosts ===
 + Run ansible
ansible-playbook -vvv -i inventory/hosts_ossl_vms ossl-buildservers.yml

PLAY [OSSL-Build | Configure build hosts] ************************************* 

GATHERING FACTS *************************************************************** 
<192.168.79.253> ESTABLISH CONNECTION FOR USER: vagrant on PORT 22 TO 192.168.79.253
fatal: [ossl-test2] => {'msg': "FAILED: (22, 'Invalid argument')", 'failed': True}

TASK: [OSSL-Common | Create folders for configuration] ************************ 
FATAL: no hosts matched or all hosts have already failed -- aborting

I’ve started investigating what is wrong, and these is result:

  • Copy know hosts to /etc/ssh/ssh_known_hosts

Inventory file:
ossl-test2 ansible_ssh_host=192.168.79.253 ansible_ssh_private_key_file=keys/vagrant ansible_connection=ssh

  • Connection from SSH:

ssh -i keys/vagrant -l vagrant 192.168.79.253

Last login: Thu Oct 24 00:27:49 2013 from 192.168.79.254
[vagrant@ossl-test2 ~]$

----> So everything is working well.

  • Connection via ansible and forcing using GlobalKnowHosts

ANSIBLE_SSH_ARGS=“-o GlobalKnownHostsFile=/etc/ssh/ssh_known_hosts” make setup-build-hosts OPTARG=“-vvv”

=== Setup build hosts ===

  • Run ansible
    ansible-playbook -vvv -i inventory/hosts_ossl_vms ossl-buildservers.yml

PLAY [OSSL-Build | Configure build hosts] *************************************

GATHERING FACTS ***************************************************************
<192.168.79.253> ESTABLISH CONNECTION FOR USER: vagrant
<192.168.79.253> EXEC [‘ssh’, ‘-tt’, ‘-q’, ‘-o’, ‘GlobalKnownHostsFile=/etc/ssh/ssh_known_hosts’, ‘-o’, ‘Port=22’, ‘-o’, ‘IdentityFile=keys/vagrant’, ‘-o’, ‘KbdInteract
iveAuthentication=no’, ‘-o’, ‘PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey’, ‘-o’, ‘PasswordAuthentication=no’, ‘-o’, ‘User=vagrant’, ‘-o’,
‘ConnectTimeout=10’, ‘192.168.79.253’, “/bin/sh -c ‘mkdir -p $HOME/.ansible/tmp/ansible-1382571109.18-256034745979242 && chmod a+rx $HOME/.ansible/tmp/ansible-13825711
09.18-256034745979242 && echo $HOME/.ansible/tmp/ansible-1382571109.18-256034745979242’”]
previous known host file not found

Message: - previous know hosts not found is displayed - but not error reported.

Run ansible without any options for SSH - causing that ansible is asking for SSH Key (displaying prompt - yes / no). IMHO should read global known_hosts file

Creating file /var/lib/jenking/.ssh/known_hosts allowed jenkins work without any problems.

Summary:

  • It looks like ansible is not reading global SSH known hosts files (when connection setup to ssh or paramiko)
  • It looks that passing option for ssh related with global knowhosts generate message: previous known host file not found - but connection can’t be established with hosts.

Goal:

  • Use ansible under jenkins without adding manually know hosts to jenkins users - just by creating global known hosts or by passing SSH parameters related with KnownHosts

Best regards,
Marcin Praczko

2

“- It looks like ansible is not reading global SSH known hosts files (when connection setup to ssh or paramiko)”

Yep, this is entirely true. Ansible doesn’t know that you passed this parameter so it’s not looking to see whether it should validate based on it.

Ansible looks ahead to see if it’s in the file to know whether it needs to lock to ask for a prompt.

However, the option you gave will still go to openssh and it should not prompt you if already there however it’s going to not parallelize properly either.

The answer here is for us to look at SSH options used and also look in the other file specified.

I’d consider this a bit of a minor one, but please file this in github and we can take care of it.

If you don’t pass this option in SSH options you also won’t have that problem and it will use the user known hosts file.

Some folks (if you understand the implications) will also consider disabling host key checking in ansible.cfg if it’s a controlled environment.

3

Hi Michael,

Thank you for reply about this. I will raise a github issue for this.

I’d consider this a bit of a minor one, but please file this in github and we can take care of it.

Not quite sure whether this is minor one. Especially when Ansible is integrated with CM - like Jenkins.
Jenkins can run many tasks as the same time. If task needs create known hosts files in $HOME for each task. Is easy to get situation that file will be overwritten by each task and can work differently.
For example:

  • ansible must configure 200 hosts. Pressing YES 200 times for first connection is really, really slow and not nice. Personally I don’t see readon why one should ignore warning about SSH keys - with this solution there is no way to detect whether connection is trusted or not (for example Men in the middle, etc).
  • manual setup know hosts is also not “fun” especially in ‘cloud’ word, when VMs can be created dynamically and all SSH finger prints should be managed dynamically as well (Destroying VMs which are no longer use should remove entries from know hosts files).

So IMHO - it would be very nice that some local file can be created and ansible is able read from this file. This way allow jenkins (CI) quickly run a lot of tasks in the same time wihtout impacting each other.

But, thank you for help with this.
Going to raise issue on github.

Best regards,
Marcin Praczko

4

“So IMHO - it would be very nice that some local file can be created”

Perhaps it could be named ~/.ssh/known_hosts :stuck_out_tongue:

5

Hi,

I totally agree that this filename is valid for SSH know hosts for human being - but not for automation.

Please imagine following scenarios:

  • 1: Ansible must connect with 300 hosts to mange them (hosted on cloud - dynamically created - quite often).
  • 2: Ansible must connect with another 100 hosts in totally differnet network - and totally different playbooks are used.
  • 3: Ansible doens’t know anything about know hosts at all.
  • 4: A lot of nodes are destroying on monthly basics and replaced with never version.

So when ansible is connecting first time - there is 400 times required press YES (I am not speaking about ignoring knowhosts - this is against SSH secure transfer).

  • We managed to have script which scans all hosts and generates knowhosts files - this is working well.
  • Now would be very nice to:
  1. Create global knowhosts files - that every user / software connecting from this host is able connect with all above nodes with valid knowhosts.
  2. Create some local file with knowhosts for automated testing.
    a) For example both tasks can be integrated to Jenkins - now Jenkins is working on ‘jenkins’ user. Each time I must somehow generate knowhows to /var/lib/jenkins/.ssh/known_hosts - which is not easy when some tasks are running in the same time.
    b) So now having local knowhosts file (not called ~/.ssh/.known_hosts) will allow run this without any problems.
    c) Lets say that one of DevOp must checkout out code and work on this, again - this DevOps must mix knowhosts which he already have with this knowhosts which are created during connection with these 400 nodes. Additional manual work :frowning:
    So having local file which ansible can read will allow quickly: SCM up / RUN with the same parameters as is run on jenkins or another automation solution. This should allow generate knowhosts before run ansible and then manage hosts :slight_smile:

But thank you very much for hits :slight_smile:

Best regards,
Marcin Praczko

6

You can run keyscan in this case to prepopulate if you like.