# Steps for ssh forwarding? **URL:** \ **Category:** Ansible Project\ **Created:** [February 2, 2015, 6:38pm UTC](https://forum.ansible.com/t/steps-for-ssh-forwarding/17422 "2015-02-02T18:38:09Z")\ **Posts on this page:** 16\ **Page:** 1
**Author:** ![Ananda\_Debnath](https://avatars.discourse-cdn.com/v4/letter/a/258eb7/32.png) [@Ananda\_Debnath](https://forum.ansible.com/u/Ananda_Debnath)\ **Post date:** [February 2, 2015, 6:38pm UTC](https://forum.ansible.com/t/steps-for-ssh-forwarding/17422/1 "2015-02-02T18:38:09Z")
Hi Ansiblers, I have an Ansible controller machine that contains my playbooks, custom libraries, key files etc. I have an inventory of host machines and CentOS VM’s that run on these host machines. The VMs aren’t directly accessible from the Ansible box. To get to each VM manually, I need to ssh into each host machine and then ssh into the specific VM. While in a shell on the Ansible machine, I can reach the VM’s by using: ssh -o “ForwardAgent=yes” -tt [king@56.66.3.10](mailto:king@56.66.3.10) ssh [redis\_user@10.0.0.1](mailto:redis_user@10.0.0.1) I.e. the chain of access is _Ansible-machine → dom0 → vm_ I’d like to specify the VMs in my inventory file - but unsure what all things need to be set in ansible.cfg, ~/.ssh/config etc. Inventory: [dom0-host] 56.66.3.10 [dom0-host:vars] ansible\_ssh\_user=king ansible\_ssh\_private\_key\_file=/home/ansible\_user/.ssh/dom0\_id\_rsa.private\_key [redis-vm] _# Can only access this from dom0-host_ 10.0.0.1 ansible\_ssh\_user=redis\_user ansible\_ssh\_pass=“reallybadidea” [web-vm] _# Can only access this from dom0-host_ 10.0.0.2 ansible\_ssh\_user=redis\_user ansible\_ssh\_pass=“anotherbadidea” I tried googling around, but couldn’t find a summary of the steps that would allow me to do this. I’m hoping this is possible and that someone can enumerate the steps here. Many thanks! Ananda ---
**Author:** ![Giovanni\_Tirloni](https://avatars.discourse-cdn.com/v4/letter/g/71c47a/32.png) [@Giovanni\_Tirloni](https://forum.ansible.com/u/Giovanni_Tirloni)\ **Post date:** [February 2, 2015, 7:08pm UTC](https://forum.ansible.com/t/steps-for-ssh-forwarding/17422/2 "2015-02-02T19:08:43Z")
Your scenario seems similar to having a bastion host so the SSH ProxyCommand directive might help. [https://www.google.com/search?q=ansible+bastion+host](https://www.google.com/search?q=ansible+bastion+host) Giovanni ---
**Author:** ![Ananda\_Debnath](https://avatars.discourse-cdn.com/v4/letter/a/258eb7/32.png) [@Ananda\_Debnath](https://forum.ansible.com/u/Ananda_Debnath)\ **Post date:** [February 3, 2015, 4:06am UTC](https://forum.ansible.com/t/steps-for-ssh-forwarding/17422/3 "2015-02-03T04:06:21Z")
Many thanks! After a few hours of wresting with this and my setup, and a typo that my eyes kept skimming over I finally got this working. Thanks again for point it out to me Giovanni! ---
**Author:** ![Ananda\_Debnath](https://avatars.discourse-cdn.com/v4/letter/a/258eb7/32.png) [@Ananda\_Debnath](https://forum.ansible.com/u/Ananda_Debnath)\ **Post date:** [February 13, 2015, 10:51pm UTC](https://forum.ansible.com/t/steps-for-ssh-forwarding/17422/4 "2015-02-13T22:51:48Z")
Does anyone know of an example to do this using **password** authentication? My inventory file has ansible\_ssh\_user and ansible\_ssh\_pass values set and my ssh config file has password authentication set to yes. my.ssh.config Host 56.66.3.10 User king HostName 56.66.3.10 ProxyCommand none BatchMode yes PasswordAuthentication yes _# Also tried ssh -aY_ _[king@56.66.3.10](mailto:king@56.66.3.10) ‘nc -w 14400 %h %p’_ Host \* ServerAliveInterval 60 TCPKeepAlive yes ProxyCommand ssh -AY [king@56.66.3.10](mailto:king@56.66.3.10) ‘nc -w 14400 %h %p’ ControlMaster auto ControlPath ~/.ssh/mux-%r@%h:%p ControlPersist 15s PasswordAuthentication yes But this isn’t helping. I have a test role that applies to 56.66.3.10 first and then other roles to the ones behind the bastion - redis-vm, etc. Neither the bastion nor the vms behind it will let me through with passwords. The ansible.cfg file looks like: [defaults] host\_key\_checking = False _# Tried both with and without this_ transport=ssh [ssh\_connection] ssh\_args = -F my.ssh.config scp\_if\_ssh = True control\_path = ~/.ssh/mux-%%r@%%h:%%p The tail of the bastion login failure reads: debug2: key: /home/adebnath/.ssh/id\_rsa ((nil)), debug2: key: /home/adebnath/.ssh/id\_dsa ((nil)), debug2: key: /home/adebnath/.ssh/id\_ecdsa ((nil)), debug2: key: /home/adebnath/.ssh/id\_ed25519 ((nil)), debug1: Authentications that can continue: publickey,password debug3: start over, passed a different list publickey,password debug3: preferred debug1: No more authentication methods to try. Permission denied (publickey,password). ---
**Author:** ![Michael\_Peters](https://avatars.discourse-cdn.com/v4/letter/m/c2a13f/32.png) [@Michael\_Peters](https://forum.ansible.com/u/Michael_Peters)\ **Post date:** [February 13, 2015, 11:05pm UTC](https://forum.ansible.com/t/steps-for-ssh-forwarding/17422/5 "2015-02-13T23:05:35Z")
Not the answer you're looking for, but why don't you just use ssh keys? It's some minor work upfront with huge security and automation benefits easier. ---
**Author:** ![Ananda\_Debnath](https://avatars.discourse-cdn.com/v4/letter/a/258eb7/32.png) [@Ananda\_Debnath](https://forum.ansible.com/u/Ananda_Debnath)\ **Post date:** [February 14, 2015, 4:10pm UTC](https://forum.ansible.com/t/steps-for-ssh-forwarding/17422/6 "2015-02-14T16:10:11Z")
Thanks for looking. There are too many current processes dependent on passwords that I’m migrating to Ansible - while converting to keys is partly underway, it won’t be complete for a while. There’s also a second bootstrapping problem. I’m using Ansible to run baremetal bringup scripts on xen hosts (doing double duty as the jumpbox/bastion host above) which in turn creates the VM’s from an image. I cannot bake keys into the image and need to get them on there after they boot up. Hence - a chicken and egg problem. How would I get the keys onto the VMs? Xen hosts have numerous problems running Ansible scripts directly because they run an old version of Python - so I don’t think I can call the authorized\_key module on the host and have it inject them into the VMs. The action would need to be triggered outside in a machine/vm that supports Ansible - which in turn would need to tunnel into the VMs to do just this - hence chicken and egg. Ultimately, the key pairs would be created and injected into the images instead of the passwords we do today - but as I mention above, this won’t be for a while. ---
**Author:** ![Ananda\_Debnath](https://avatars.discourse-cdn.com/v4/letter/a/258eb7/32.png) [@Ananda\_Debnath](https://forum.ansible.com/u/Ananda_Debnath)\ **Post date:** [February 15, 2015, 6:13am UTC](https://forum.ansible.com/t/steps-for-ssh-forwarding/17422/7 "2015-02-15T06:13:46Z")
I did manage to get it to work - but it’s really ugly 😕 I had to change the ProxyCommand directive in the ssh config for the wildcard section to: ProxyCommand sshpass -p ‘reallybadpassword’ ssh -A [root@56.66.3.10](mailto:root@56.66.3.10) ‘nc -w 14400 %h %p’ I guess Ansible has no way of overriding the ProxyCommand specified in the ssh\_config file in order to use sshpass with the password specified in the inventory file? ---
**Author:** ![Michael\_Spiegle](https://avatars.discourse-cdn.com/v4/letter/m/8797f3/32.png) [@Michael\_Spiegle](https://forum.ansible.com/u/Michael_Spiegle)\ **Post date:** [February 16, 2015, 12:11am UTC](https://forum.ansible.com/t/steps-for-ssh-forwarding/17422/8 "2015-02-16T00:11:58Z")
As an additional datapoint, here’s a brief summary of how I deal with this. To complicate matters, my machines are split across various labs in different locations which each have their own bastion/jumpbox. I use ssh keys sometimes, and hard coded passwords for some other machines: ansible.cfg: ` [defaults] hostfile = hosts error\_on\_undefined\_vars = True host\_key\_checking = False transport = ssh jinja2\_extensions = jinja2.ext.do [ssh\_connection] ssh\_args = -F ssh.config pipelining = True ` ssh.config (referenced in ansible.cfg). Note that if you have a new enough version of ssh, you can use the -W flag instead of nc: ` #jumpboxes first (most specific hosts first) Host jumpbox01 10.1.0.10 ControlMaster yes ControlPath ~/.ssh/master-%r@jumpbox01:%p StrictHostkeyChecking no ProxyCommand none Host jumpbox02 10.2.0.10 ControlMaster yes ControlPath ~/.ssh/master-%r@jumpbox02:%p StrictHostkeyChecking no ProxyCommand none Host jumpbox03 10.3.0.10 ControlMaster yes ControlPath ~/.ssh/master-%r@jumpbox03:%p StrictHostkeyChecking no ProxyCommand none # groups of machines that can be accessed by the above jumpboxes Host _.west.domain.com 10.1.0._ ControlMaster no ProxyCommand ssh -S ~/.ssh/master-\*@jumpbox01:%p remote nc %h %p Host _.central.domain.com 10.2.0._ ControlMaster no ProxyCommand ssh -S ~/.ssh/master-\*@jumpbox02:%p remote nc %h %p Host _.east.domain.com 10.3.0._ ControlMaster no ProxyCommand ssh -S ~/.ssh/master-\*@jumpbox03:%p remote nc %h %p # this makes ansible faster by reusing connections Host \* ControlMaster auto ControlPersist 300s ControlPath ~/.ssh/ansible-%r@%h:%p ` Once I have those configs setup, I have to run the following to establish a tunnel to a jumpbox/bastion before I can run ansible: ` $ ssh -F ssh.config -fN user@jumpbox01 ` When I run the above, it asks for the password (or uses my SSH key), then SSH goes into the background and then the tunnel is established. I do all of my deployments this way by creating a Jenkins job that establishes the tunnel, runs ansible, then tears down the tunnel using something like: `ssh -O exit -TS ~/.ssh/path-to-socket` Some of the jumpboxes use dumb passwords, some of them use keys, and one of them requires an RSA token (2-factor auth). For the RSA machine, my Jenkins job presents the user with a form that has 2 fields: 2-Factor Username, and 2-Factor Passcode. The passcode is generated by an RSA token keyfob (or smartphone app). In order to make this work, I had to write a custom expect script because the SSH prompt for the RSA token reads “Enter PASSCODE” instead of “Password” which is what sshpass is hardcoded to look for. Here’s my expect script to catch all the variations: ` #!/usr/bin/env expect set timeout 30 set userhost [lindex $argv 0] spawn ssh -fN -F ssh.config $userhost expect { “Enter PASSCODE:” { send “$env(SSH\_PASSWORD)\n” send “\n” } “Password:” { send “$env(SSH\_PASSWORD)\n” send “\n” } “password:” { send “$env(SSH\_PASSWORD)\n” send “\n” } } sleep 5 ` I need the sleep 5 at the end of the script as a hack because the jumpbox with RSA token don’t establish the control socket until a few seconds after the login happens. If my script exits too soon, then the tunnel won’t get established. I’m not sure how to properly deal with this. Like I said, some of the machines that I run ansible on have hardcoded passwords and it works fine when ssh\_user and ssh\_pass is set as facts for your host. ---
**Author:** ![Michael\_Spiegle](https://avatars.discourse-cdn.com/v4/letter/m/8797f3/32.png) [@Michael\_Spiegle](https://forum.ansible.com/u/Michael_Spiegle)\ **Post date:** [February 16, 2015, 12:14am UTC](https://forum.ansible.com/t/steps-for-ssh-forwarding/17422/9 "2015-02-16T00:14:54Z")
About half of my machines are in Amazon/EC2. In order to solve the chicken/egg problem, I write out a “user\_data” script which installs some SSH keys for me to the root user of the VM upon first boot. This allows me to run my initial bootstrap and get the machine joined to the domain, then I can optionally remove the root keys. This is all done with Ansible. ---
**Author:** ![Ananda\_Debnath](https://avatars.discourse-cdn.com/v4/letter/a/258eb7/32.png) [@Ananda\_Debnath](https://forum.ansible.com/u/Ananda_Debnath)\ **Post date:** [February 17, 2015, 1:39pm UTC](https://forum.ansible.com/t/steps-for-ssh-forwarding/17422/10 "2015-02-17T13:39:10Z")
Many thanks Michael for the detailed writeup! I’m going to experiment with my setup (quite similar to your’s - multiple jumpboxes in multiple locations, some AWS usage as well) to see how much of your approach I can adapt to mine. ---
**Author:** ![Benny\_Kusman](https://avatars.discourse-cdn.com/v4/letter/b/4da419/32.png) [@Benny\_Kusman](https://forum.ansible.com/u/Benny_Kusman)\ **Post date:** [April 5, 2018, 10:20am UTC](https://forum.ansible.com/t/steps-for-ssh-forwarding/17422/11 "2018-04-05T10:20:14Z")
this is a great step. im able to establish the tunneling with the jumphost. but i was wondering, what did you put in the hosts filfe ? im still not able to reach the server ---
**Author:** ![Michael\_Spiegle](https://avatars.discourse-cdn.com/v4/letter/m/8797f3/32.png) [@Michael\_Spiegle](https://forum.ansible.com/u/Michael_Spiegle)\ **Post date:** [April 5, 2018, 12:46pm UTC](https://forum.ansible.com/t/steps-for-ssh-forwarding/17422/12 "2018-04-05T12:46:14Z")
By “hosts” file, do you mean /etc/hosts or the hosts in ssh.config? Also, if you just run SSH by hand to login to a remote host, what happens? Ex: $ ssh -F ssh.config -fN user@some\_jumpbox $ ssh -F ssh.config user@some\_server\_behind\_jumpbox ---
**Author:** ![Benny\_Kusman](https://avatars.discourse-cdn.com/v4/letter/b/4da419/32.png) [@Benny\_Kusman](https://forum.ansible.com/u/Benny_Kusman)\ **Post date:** [April 9, 2018, 7:18am UTC](https://forum.ansible.com/t/steps-for-ssh-forwarding/17422/13 "2018-04-09T07:18:17Z")
Hi ssh -F ssh.config -fN user@some\_jumpbox → im able to establish this and send it back to the background ssh -F ssh.config user@someserverbehindjumpbox → does not work. the message is: ssh\_exchange\_identification: Connection closed by remote host the “hosts” im referring to is in the ssh.config. This is my ssh.config ---
**Author:** ![Benny\_Kusman](https://avatars.discourse-cdn.com/v4/letter/b/4da419/32.png) [@Benny\_Kusman](https://forum.ansible.com/u/Benny_Kusman)\ **Post date:** [April 9, 2018, 7:30am UTC](https://forum.ansible.com/t/steps-for-ssh-forwarding/17422/14 "2018-04-09T07:30:05Z")
This is the -vvv output: [root@WW-GVXQLC2 ansible]# ssh -F ssh.config bkusman@serverbehindjumpbox -p 670 -vvv OpenSSH\_7.4p1, OpenSSL 1.0.2k-fips 26 Jan 2017 debug1: Reading configuration data ssh.config debug1: ssh.config line 28: Applying options for \* debug1: Executing proxy command: exec ssh -S ~/.ssh/master-\*@jumpboxserver:670 -W serverbehindjumpbox:670 debug1: permanently\_set\_uid: 0/0 debug1: permanently\_drop\_suid: 0 debug1: key\_load\_public: No such file or directory debug1: identity file /root/.ssh/id\_rsa type -1 debug1: key\_load\_public: No such file or directory debug1: identity file /root/.ssh/id\_rsa-cert type -1 debug1: key\_load\_public: No such file or directory debug1: identity file /root/.ssh/id\_dsa type -1 debug1: key\_load\_public: No such file or directory debug1: identity file /root/.ssh/id\_dsa-cert type -1 debug1: key\_load\_public: No such file or directory debug1: identity file /root/.ssh/id\_ecdsa type -1 debug1: key\_load\_public: No such file or directory debug1: identity file /root/.ssh/id\_ecdsa-cert type -1 debug1: key\_load\_public: No such file or directory debug1: identity file /root/.ssh/id\_ed25519 type -1 debug1: key\_load\_public: No such file or directory debug1: identity file /root/.ssh/id\_ed25519-cert type -1 debug1: Enabling compatibility mode for protocol 2.0 debug1: Local version string SSH-2.0-OpenSSH\_7.4 usage: ssh [-1246AaCfGgKkMNnqsTtVvXxYy] [-b bind\_address] [-c cipher\_spec] [-D [bind\_address:]port] [-E log\_file] [-e escape\_char] [-F configfile] [-I pkcs11] [-i identity\_file] [-J [user@]host[:port]] [-L address] [-l login\_name] [-m mac\_spec] [-O ctl\_cmd] [-o option] [-p port] [-Q query\_option] [-R address] [-S ctl\_path] [-W host:port] [-w local\_tun[:remote\_tun]] [user@]hostname [command] ssh\_exchange\_identification: Connection closed by remote host ---
**Author:** ![mspiegle](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/mspiegle/32/2395_2.png) [@mspiegle](https://forum.ansible.com/u/mspiegle)\ **Post date:** [April 9, 2018, 1:48pm UTC](https://forum.ansible.com/t/steps-for-ssh-forwarding/17422/15 "2018-04-09T13:48:30Z")
I think this is because the SSH command always expects a hostname even if there isn’t a hostname to use. In your ssh -vvv debug output, you can see that every single option is inside of [optional] brackets EXCEPT for the hostname. SSH won’t actually use this hostname for anything, it just wants some dumb string in that field. In my config, I use the string “remote” like the following. Note that the host “remote” doesn’t exist and you’d probably receive an error under normal circumstances: ---
**Author:** ![Benny\_Kusman](https://avatars.discourse-cdn.com/v4/letter/b/4da419/32.png) [@Benny\_Kusman](https://forum.ansible.com/u/Benny_Kusman)\ **Post date:** [April 10, 2018, 4:09am UTC](https://forum.ansible.com/t/steps-for-ssh-forwarding/17422/16 "2018-04-10T04:09:11Z")
Michael, you are SO RIGHT! now the tunneling works after your clue! completely missed out this “ **remote** ”