delegate_to running on unexpected host

Hi all,

My understanding of delegate_to is that it runs the task on the designated host, once for each host in the current group, and with the current host’s variables.

I have this:

- hosts: 127.0.0.1
  connection: local
  - tasks:
    [ … stuff … ]
    - name: Copy override file for database information
      template: >
        src=templates/user_data.json
        dest={{ app_exec_dir }}/core/AWS_OVERRIDE.json
      delegate_to: "{{ retrieved_instance.public_ip }}"

I see this, where <proper host> is the correct IP address:

TASK: [Copy override file for database information] ***************************
<proper host> EXEC ['/bin/sh', '-c', 'mkdir -p $HOME/.ansible/tmp/ansible-tmp-1399297955.21-265740010111281 && chmod a+rx $HOME/.ansible/tmp/ansible-tmp-1399297955.21-265740010111281 && echo $HOME/.ansible/tmp/ansible-tmp-1399297955.21-265740010111281']
<proper host> EXEC ['/bin/sh', '-c', u'rc=0; [ -r "/srv/sp/web/core/AWS_OVERRIDE.json" ] || rc=2; [ -f "/srv/sp/web/core/AWS_OVERRIDE.json" ] || rc=1; [ -d "/srv/sp/web/core/AWS_OVERRIDE.json" ] && echo 3 && exit 0; (/usr/bin/md5sum /srv/sp/web/core/AWS_OVERRIDE.json 2>/dev/null) || (/sbin/md5sum -q /srv/sp/web/core/AWS_OVERRIDE.json 2>/dev/null) || (/usr/bin/digest -a md5 /srv/sp/web/core/AWS_OVERRIDE.json 2>/dev/null) || (/sbin/md5 -q /srv/sp/web/core/AWS_OVERRIDE.json 2>/dev/null) || (/usr/bin/md5 -n /srv/sp/web/core/AWS_OVERRIDE.json 2>/dev/null) || (/bin/md5 -q /srv/sp/web/core/AWS_OVERRIDE.json 2>/dev/null) || (/usr/bin/csum -h MD5 /srv/sp/web/core/AWS_OVERRIDE.json 2>/dev/null) || (/bin/csum -h MD5 /srv/sp/web/core/AWS_OVERRIDE.json 2>/dev/null) || (echo "${rc} /srv/sp/web/core/AWS_OVERRIDE.json")']
<proper host> PUT /var/folders/m4/mk3x6q_50dj_50jpx0763lm80000gn/T/tmpl9p7QM TO /Users/anderson/.ansible/tmp/ansible-tmp-1399297955.21-265740010111281/source
<proper host> PUT /var/folders/m4/mk3x6q_50dj_50jpx0763lm80000gn/T/tmpB7_vdY TO /Users/anderson/.ansible/tmp/ansible-tmp-1399297955.21-265740010111281/copy
<proper host> EXEC ['/bin/sh', '-c', '/usr/bin/python /Users/anderson/.ansible/tmp/ansible-tmp-1399297955.21-265740010111281/copy; rm -rf /Users/anderson/.ansible/tmp/ansible-tmp-1399297955.21-265740010111281/ >/dev/null 2>&1']
failed: [127.0.0.1] => {"failed": true}
msg: Destination /srv/sp/web/core not writable

The directory does exist on the delegate_to host:

(sp_devops)GoatRodeo:devops anderson$ ssh -i keys/hoopdy.id_rsa ubuntu@<proper host>
ubuntu@ip-10-2-1-174:~$ ls /srv/sp/web/core
[… stuff …]

If I add the directory to the *local* machine running Ansible, the script finds it. This is telling me that the template command is actually trying to copy to the local machine and not the delegate_to machine.

If this is not what delegate_to is for, is there another way to run a single task on a single host not in the current group without having to pop all the way back up to the playbook context and add another host: section?

Thanks,
-scott

delegate_to does work, I suspect your issues stem from the variable not being defined or being empty, delegate_to will fallback to localhost.

Brian Coca

“My understanding of delegate_to is that it runs the task on the designated host, once for each host in the current group, and with the current host’s variables.”

Delegate_to runs the given step on the delegated host, instead of the one currently being referenced in the host loop.

" I suspect your issues stem from the variable not being defined or being empty, delegate_to will fallback to localhost."

This really shouldn’t be the case, and I would consider that a bug if so.

It should be easy to test that independent of the above example, however.

Hi all,

My understanding of delegate_to is that it runs the task on the designated host, once for each host in the current group, and with the current host’s variables.

msg: Destination /srv/sp/web/core not writable

The directory does exist on the delegate_to host:

(sp_devops)GoatRodeo:devops anderson$ ssh -i keys/hoopdy.id_rsa ubuntu@
ubuntu@ip-10-2-1-174:~$ ls /srv/sp/web/core
[… stuff …]

If I add the directory to the local machine running Ansible, the script finds it. This is telling me that the template command is actually trying to copy to the local machine and not the delegate_to machine.

It’s not saying that the directory doesn’t exist, it’s saying that it is not writable, do you have write permission to /srv/sp/web/core on the delegate_to machine? ls -ld /srv/sp/web/core should show what I’m interested in…

Adam

Imagine that was filled in with an actual IP in the log (I removed it for Teh Securityz).

<54.1.2.3> PUT /var/folders/m4/mk3x6q_50dj_50jpx0763lm80000gn/T/tmpB7_vdY TO /Users/anderson/.ansible/tmp/ansible-tmp-1399297955.21-265740010111281/copy

The task is currently running with a host of 127.0.0.1. <54.1.2.3> would not be there if the variable being used for delegate_to were empty.

Another clue: /Users/anderson only exists on my local machine, not on the remote machine (which is 12.04 LTS, not OSX). The task is running a PUT to the local machine, not 54.1.2.3.

Regards,
-scott

The directory exists and is writable (it’s owned by ubuntu) on the delegate_to machine. It does not exist on localhost.

As an additional test, I added the directory on localhost and made it writable, at which point the task no longer failed indicating that it is running on localhost and not on the delegate_to IP:

(sp_devops)GoatRodeo:devops anderson$ sudo chmod aoug+w /srv/sp/web/core
Password:
(sp_devops)GoatRodeo:devops anderson$ ls -Faxl /srv/sp/web/core
total 0
drwxrwxrwx 2 root wheel 68 May 6 14:14 ./
drwxr-xr-x 3 root wheel 102 May 6 14:14 …/

TASK: [debug msg=“retrieved_instance.public_ip is {{retrieved_instance.public_ip}}”] ***
ok: [127.0.0.1] => {
“msg”: “retrieved_instance.public_ip is 54.215.196.1”
}

TASK: [Copy override file for database information] ***************************
<54.215.196.1> EXEC [‘/bin/sh’, ‘-c’, ‘mkdir -p $HOME/.ansible/tmp/ansible-tmp-1399400706.01-38026459986463 && chmod a+rx $HOME/.ansible/tmp/ansible-tmp-1399400706.01-38026459986463 && echo $HOME/.ansible/tmp/ansible-tmp-1399400706.01-38026459986463’]
<54.215.196.1> EXEC [‘/bin/sh’, ‘-c’, u’rc=0; [ -r “/srv/sp/web/core/AWS_OVERRIDE.json” ] || rc=2; [ -f “/srv/sp/web/core/AWS_OVERRIDE.json” ] || rc=1; [ -d “/srv/sp/web/core/AWS_OVERRIDE.json” ] && echo 3 && exit 0; (/usr/bin/md5sum /srv/sp/web/core/AWS_OVERRIDE.json 2>/dev/null) || (/sbin/md5sum -q /srv/sp/web/core/AWS_OVERRIDE.json 2>/dev/null) || (/usr/bin/digest -a md5 /srv/sp/web/core/AWS_OVERRIDE.json 2>/dev/null) || (/sbin/md5 -q /srv/sp/web/core/AWS_OVERRIDE.json 2>/dev/null) || (/usr/bin/md5 -n /srv/sp/web/core/AWS_OVERRIDE.json 2>/dev/null) || (/bin/md5 -q /srv/sp/web/core/AWS_OVERRIDE.json 2>/dev/null) || (/usr/bin/csum -h MD5 /srv/sp/web/core/AWS_OVERRIDE.json 2>/dev/null) || (/bin/csum -h MD5 /srv/sp/web/core/AWS_OVERRIDE.json 2>/dev/null) || (echo “${rc} /srv/sp/web/core/AWS_OVERRIDE.json”)‘]
<54.215.196.1> PUT /var/folders/m4/mk3x6q_50dj_50jpx0763lm80000gn/T/tmp68MzNT TO /Users/anderson/.ansible/tmp/ansible-tmp-1399400706.01-38026459986463/source
<54.215.196.1> PUT /var/folders/m4/mk3x6q_50dj_50jpx0763lm80000gn/T/tmpZah9PU TO /Users/anderson/.ansible/tmp/ansible-tmp-1399400706.01-38026459986463/copy
<54.215.196.1> EXEC [’/bin/sh’, ‘-c’, ‘/usr/bin/python /Users/anderson/.ansible/tmp/ansible-tmp-1399400706.01-38026459986463/copy; rm -rf /Users/anderson/.ansible/tmp/ansible-tmp-1399400706.01-38026459986463/ >/dev/null 2>&1’]
changed: [127.0.0.1] => {“changed”: true, “dest”: “/srv/sp/web/core/AWS_OVERRIDE.json”, “gid”: 20, “group”: “staff”, “md5sum”: “f1f374872c6ddab3a2b38c1176df2a0a”, “mode”: “0644”, “owner”: “anderson”, “size”: 312, “src”: “/Users/anderson/.ansible/tmp/ansible-tmp-1399400706.01-38026459986463/source”, “state”: “file”, “uid”: 501}

The task is clearly running on 127.0.0.1 and not 54.215.196.1. The “is not writable” error apparently is returned for both non-existent and non-writable directories.

Regards,
-scott

For extra special fun I made up an IP address and used it in delegate_to:

  • name: Copy override file for database information
    template: >
    src=templates/user_data.json
    dest={{ aws_settings_override_file_path }}
    delegate_to: 34.142.33.2

(sp_devops)GoatRodeo:devops anderson$ ping 34.142.33.2
PING 34.142.33.2 (34.142.33.2): 56 data bytes
Request timeout for icmp_seq 0

TASK: [Copy override file for database information] ***************************
<34.142.33.2> EXEC [‘/bin/sh’, ‘-c’, ‘mkdir -p $HOME/.ansible/tmp/ansible-tmp-1399401156.95-113268015974642 && chmod a+rx $HOME/.ansible/tmp/ansible-tmp-1399401156.95-113268015974642 && echo $HOME/.ansible/tmp/ansible-tmp-1399401156.95-113268015974642’]
<34.142.33.2> EXEC [‘/bin/sh’, ‘-c’, u’rc=0; [ -r “/srv/sp/web/core/AWS_OVERRIDE.json” ] || rc=2; [ -f “/srv/sp/web/core/AWS_OVERRIDE.json” ] || rc=1; [ -d “/srv/sp/web/core/AWS_OVERRIDE.json” ] && echo 3 && exit 0; (/usr/bin/md5sum /srv/sp/web/core/AWS_OVERRIDE.json 2>/dev/null) || (/sbin/md5sum -q /srv/sp/web/core/AWS_OVERRIDE.json 2>/dev/null) || (/usr/bin/digest -a md5 /srv/sp/web/core/AWS_OVERRIDE.json 2>/dev/null) || (/sbin/md5 -q /srv/sp/web/core/AWS_OVERRIDE.json 2>/dev/null) || (/usr/bin/md5 -n /srv/sp/web/core/AWS_OVERRIDE.json 2>/dev/null) || (/bin/md5 -q /srv/sp/web/core/AWS_OVERRIDE.json 2>/dev/null) || (/usr/bin/csum -h MD5 /srv/sp/web/core/AWS_OVERRIDE.json 2>/dev/null) || (/bin/csum -h MD5 /srv/sp/web/core/AWS_OVERRIDE.json 2>/dev/null) || (echo “${rc} /srv/sp/web/core/AWS_OVERRIDE.json”)‘]
<34.142.33.2> PUT /var/folders/m4/mk3x6q_50dj_50jpx0763lm80000gn/T/tmpBAaq73 TO /Users/anderson/.ansible/tmp/ansible-tmp-1399401156.95-113268015974642/file
<34.142.33.2> EXEC [’/bin/sh’, ‘-c’, ‘/usr/bin/python /Users/anderson/.ansible/tmp/ansible-tmp-1399401156.95-113268015974642/file; rm -rf /Users/anderson/.ansible/tmp/ansible-tmp-1399401156.95-113268015974642/ >/dev/null 2>&1’]
ok: [127.0.0.1] => {“changed”: false, “gid”: 20, “group”: “staff”, “mode”: “0644”, “owner”: “anderson”, “path”: “/srv/sp/web/core/AWS_OVERRIDE.json”, “size”: 312, “state”: “file”, “uid”: 501}

:slight_smile:

Regards,
-scott