Unable to ssh onto newly created ec2 instance

Archives Ansible Project
1

Hi all

I’m trying to get a playbook to create an ec2 instance, run a bunch of stuff and then delete the instance based on the ec2 example in the docs. I’m having an issue where after it waits for ssh to come up, it fails to ssh into the host. Manually sshing to the instance as root works just fine from the cmd line after the playbook fails.

Here is the output with -vvvv http://pastebin.com/FQY53BB6

My hosts file has the following:

[localhost]
127.0.0.1 ansible_ssh_user=root

Any ideas what I’m doing wrong?

Thanks,
Marc

2

From the log it looks like it is cycling through your public keys and failing

  1. debug1: Offering RSA public key: /root/.ssh/id_rsa

  2. debug3: send_pubkey_test

  3. debug2: we sent a publickey packet, wait for reply

  4. debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password

  5. debug1: Trying private key: /root/.ssh/id_dsa

  6. debug3: no such identity: /root/.ssh/id_dsa

  7. debug1: Trying private key: /root/.ssh/id_ecdsa

  8. debug3: no such identity: /root/.ssh/id_ecdsa

  9. debug2: we did not send a packet, disable method

  10. debug1: No more authentication methods to try.

When you ssh to your instance what key are you using?
Try $ ssh -i /root/.ssh/ and make sure that works if you believe one of the keys it tried is valid

I don’t believe it’s the problem but you will need to set “host_key_checking = False” in your ansible.cfg for 1.2+ since your new instance won’t be in your known_hosts file.

Finally. if you want to be more explicit, pass these options when you run the play:

$ ansible-playbook … --user -e “ansible_ssh_private_key_file=/full/path/to/your/key.pem”

-John

3

Marc,

Could we see your playbook or an anonymized snippet at least? I'm curious about how you are referencing the dynamically added hosts in your play.

Thanks,
James

4

I only have one key available:

optaplanner@optaplanner:~$ ls -l .ssh/
total 16
-rw-rw-r-- 1 optaplanner optaplanner 52 Oct 15 06:54 config
-rw------- 1 optaplanner optaplanner 1675 Oct 14 21:35 id_rsa
-rw-r–r-- 1 optaplanner optaplanner 410 Oct 14 21:35 id_rsa.pub
-rw-r–r-- 1 optaplanner optaplanner 3978 Oct 15 20:09 known_hosts

And I can ssh onto the box using that key:

optaplanner@optaplanner:~$ ssh root@54.251.71.221 -i ~/.ssh/id_rsa
Last login: Tue Oct 15 21:04:21 2013 from 54.254.54.51
[root@ip-10-128-87-77 ~]#

I tried adding “host_key_checking = False” to ansible.cfg but still get the same result. I also have “StrictHostKeyChecking no” to my ssh config so that I’m not prompted to accept the new host key every time.

Also tried running “$ ansible-playbook --user root -e “ansible_ssh_private_key_file=/home/optaplanner/.ssh/id_rsa” site.yml -vvvv” but get the same thing (see below).

GATHERING FACTS ***************************************************************
<122.248.220.172> ESTABLISH CONNECTION FOR USER: root
<122.248.220.172> EXEC [‘ssh’, ‘-tt’, ‘-vvv’, ‘-o’, ‘ControlMaster=auto’, ‘-o’, ‘ControlPersist=60s’, ‘-o’, ‘ControlPath=/home/optaplanner/.ansible/cp/ansible-ssh-%h-%p-%r’, ‘-o’, ‘StrictHostKeyChecking=no’, ‘-o’, ‘Port=22’, ‘-o’, u’IdentityFile=/home/optaplanner/.ssh/id_rsa’, ‘-o’, ‘KbdInteractiveAuthentication=no’, ‘-o’, ‘PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey’, ‘-o’, ‘PasswordAuthentication=no’, ‘-o’, ‘User=root’, ‘-o’, ‘ConnectTimeout=10’, u’122.248.220.172’, “/bin/sh -c ‘mkdir -p $HOME/.ansible/tmp/ansible-1381876732.87-94895521671279 && echo $HOME/.ansible/tmp/ansible-1381876732.87-94895521671279’”]
fatal: [122.248.220.172] => SSH encountered an unknown error. The output was:
OpenSSH_5.9p1 Debian-5ubuntu1, OpenSSL 1.0.1 14 Mar 2012
debug1: Reading configuration data /home/optaplanner/.ssh/config
debug1: /home/optaplanner/.ssh/config line 3: Applying options for *
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug1: auto-mux: Trying existing master
debug1: Control socket “/home/optaplanner/.ansible/cp/ansible-ssh-122.248.220.172-22-root” does not exist
debug2: ssh_connect: needpriv 0
debug1: Connecting to 122.248.220.172 [122.248.220.172] port 22.
debug2: fd 3 setting O_NONBLOCK
debug1: fd 3 clearing O_NONBLOCK
debug1: Connection established.
debug3: timeout: 9983 ms remain after connect
debug3: Incorrect RSA1 identifier
debug3: Could not load “/home/optaplanner/.ssh/id_rsa” as a RSA1 public key
debug1: identity file /home/optaplanner/.ssh/id_rsa type 1
debug1: Checking blacklist file /usr/share/ssh/blacklist.RSA-2048
debug1: Checking blacklist file /etc/ssh/blacklist.RSA-2048
debug1: identity file /home/optaplanner/.ssh/id_rsa-cert type -1
debug1: Remote protocol version 2.0, remote software version OpenSSH_5.3
debug1: match: OpenSSH_5.3 pat OpenSSH*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_5.9p1 Debian-5ubuntu1
debug2: fd 3 setting O_NONBLOCK
debug3: load_hostkeys: loading entries for host “122.248.220.172” from file “/home/optaplanner/.ssh/known_hosts”
debug3: load_hostkeys: loaded 0 keys
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug2: kex_parse_kexinit: ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
debug2: kex_parse_kexinit: ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com,ssh-dss-cert-v01@openssh.com,ssh-rsa-cert-v00@openssh.com,ssh-dss-cert-v00@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-rsa,ssh-dss
debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se
debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se
debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-sha2-256,hmac-sha2-256-96,hmac-sha2-512,hmac-sha2-512-96,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-sha2-256,hmac-sha2-256-96,hmac-sha2-512,hmac-sha2-512-96,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: none,zlib@openssh.com,zlib
debug2: kex_parse_kexinit: none,zlib@openssh.com,zlib
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit: first_kex_follows 0
debug2: kex_parse_kexinit: reserved 0
debug2: kex_parse_kexinit: diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
debug2: kex_parse_kexinit: ssh-rsa,ssh-dss
debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se
debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se
debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: none,zlib@openssh.com
debug2: kex_parse_kexinit: none,zlib@openssh.com
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit: first_kex_follows 0
debug2: kex_parse_kexinit: reserved 0
debug2: mac_setup: found hmac-md5
debug1: kex: server->client aes128-ctr hmac-md5 none
debug2: mac_setup: found hmac-md5
debug1: kex: client->server aes128-ctr hmac-md5 none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
debug2: dh_gen_key: priv key bits set: 130/256
debug2: bits set: 531/1024
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
debug1: Server host key: RSA 94:b2:2c:4e:9e:8e:31:3d:3f:a2:4e:7d:5d:77:e4:5c
debug3: load_hostkeys: loading entries for host “122.248.220.172” from file “/home/optaplanner/.ssh/known_hosts”
debug3: load_hostkeys: loaded 0 keys
Warning: Permanently added ‘122.248.220.172’ (RSA) to the list of known hosts.
debug2: bits set: 517/1024
debug1: ssh_rsa_verify: signature correct
debug2: kex_derive_keys
debug2: set_newkeys: mode 1
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug2: set_newkeys: mode 0
debug1: SSH2_MSG_NEWKEYS received
debug1: Roaming not allowed by server
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug2: key: /home/optaplanner/.ssh/id_rsa (0x7fe755d24a60)
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password
debug3: start over, passed a different list publickey,gssapi-keyex,gssapi-with-mic,password
debug3: preferred gssapi-with-mic,gssapi-keyex,hostbased,publickey
debug3: authmethod_lookup gssapi-with-mic
debug3: remaining preferred: gssapi-keyex,hostbased,publickey
debug3: authmethod_is_enabled gssapi-with-mic
debug1: Next authentication method: gssapi-with-mic
debug1: Unspecified GSS failure. Minor code may provide more information
Credentials cache file ‘/tmp/krb5cc_1001’ not found

debug1: Unspecified GSS failure. Minor code may provide more information
Credentials cache file ‘/tmp/krb5cc_1001’ not found

debug1: Unspecified GSS failure. Minor code may provide more information

debug1: Unspecified GSS failure. Minor code may provide more information
Credentials cache file ‘/tmp/krb5cc_1001’ not found

debug2: we did not send a packet, disable method
debug3: authmethod_lookup gssapi-keyex
debug3: remaining preferred: hostbased,publickey
debug3: authmethod_is_enabled gssapi-keyex
debug1: Next authentication method: gssapi-keyex
debug1: No valid Key exchange context
debug2: we did not send a packet, disable method
debug3: authmethod_lookup publickey
debug3: remaining preferred: ,publickey
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /home/optaplanner/.ssh/id_rsa
debug3: send_pubkey_test
debug2: we sent a publickey packet, wait for reply
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password
debug2: we did not send a packet, disable method
debug1: No more authentication methods to try.
Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).

TASK: [Download EPEL repo] ****************************************************
FATAL: no hosts matched or all hosts have already failed – aborting

PLAY RECAP ********************************************************************
to retry, use: --limit @/home/optaplanner/site.retry

122.248.220.172 : ok=0 changed=0 unreachable=1 failed=0
127.0.0.1 : ok=3 changed=1 unreachable=0 failed=0

optaplanner@optaplanner:~/optaplanner-ansible$ ssh root@122.248.220.172
[root@ip-10-135-39-36 ~]#

Regards,
Marc

5

Hi James,

Sure thing, here’s it is: