I posted this as a bug (https://github.com/ansible/ansible/issues/11102). But it was closed saying it was a discussion. Not sure why, seems like a clear bug to me. But I’m open to be dissuaded.
From the output of the 3rd task it seems that
mongodb_replica_set_hosts[0] is being interpreted as '{' which, of
course, does not resolve. This does not seem to be an array issue but
of too many nested templating attempts.
What do you think would be causing the nested templating? Something deep in variable resolution?
The third echo is the direct version of first setting the value to a fact, which works.
not sure what 'works' means in this context
“works” is that the third task delegates to the same host as the second task.
Don’t you expect the exact same output for the last two tasks? Are you seeing something that I am missing?
the task fails with e 'fatal', what looks the same is the verbose
output of what will be executed
2) ssh ... -o ConnectTimeout=10 10.0.139.56 /bin/sh -c ... 3) ssh ... -o ConnectTimeout=10 { /bin/sh -c ...
So you don’t think that the ssh ip in the 3 task should not be the same as 2?
I’m sorry Brian, but I am not all following what the miscommunication is here.
the ssh ip in task 3 is '{' which indicates that the [0] is applied
to the 'template string' and not to the resulting array you expect.
This makes the 3rd task fail as it cannot ssh to '{'.
Exactly. Is that a mistake I am making? Or a bug? I filed the bug because I don’t see a mistake on my part.
can you show that variable in a debug during execution? it does not
look like it is being set properly.
Here’s the new test:
`
-
set_fact:
mongodb_candidate_master: “{{mongodb_replica_set_hosts[0]}}”
run_once: true -
name: var groups.mongod_rs_member[0]
debug: var=groups.mongod_rs_member[0]
run_once: true -
shell: echo “HELLO groups.mongod_rs_member +++++++++++++++++”
delegate_to: “{{groups.mongod_rs_member[0]}}”
run_once: true -
name: var mongodb_candidate_master
debug: var=mongodb_candidate_master
run_once: true -
shell: echo “HELLO mongodb_candidate_master +++++++++++++++++”
delegate_to: “{{mongodb_candidate_master}}”
run_once: true -
name: var mongodb_replica_set_hosts[0]
debug: var=mongodb_replica_set_hosts[0]
run_once: true -
name: msg mongodb_replica_set_hosts[0]
debug: msg=“{{mongodb_replica_set_hosts[0]}}”
run_once: true -
debug: msg=“{{mongodb_replica_set_hosts[0]}}”
run_once: true -
shell: echo “HELLO mongodb_replica_set_hosts +++++++++++++++++”
delegate_to: “{{mongodb_replica_set_hosts[0]}}”
run_once: true
`
And the output:
`
ansible❯ ansible-playbook -i inventory/aws/staging playbooks/configure/mongod.yml --skip-tags base ops/git/master !+
PLAY [Base ansible configuration (packages)] **********************************
GATHERING FACTS ***************************************************************
ok: [10.0.137.189]
ok: [10.0.139.51]
ok: [10.0.136.46]
PLAY [Base ansible configuration (configure)] *********************************
GATHERING FACTS ***************************************************************
ok: [10.0.137.189]
ok: [10.0.139.51]
ok: [10.0.136.46]
PLAY [Base ansible configuration (users)] *************************************
GATHERING FACTS ***************************************************************
ok: [10.0.137.189]
ok: [10.0.139.51]
ok: [10.0.136.46]
PLAY [Configure mongodb replicaset cluster] ***********************************
GATHERING FACTS ***************************************************************
ok: [10.0.136.46]
ok: [10.0.137.189]
ok: [10.0.139.51]
TASK: [debug var=groups.mongod_rs_member] *************************************
ok: [10.0.136.46] => {
“var”: {
“groups.mongod_rs_member”: [
“10.0.136.46”,
“10.0.139.51”,
“10.0.137.189”
]
}
}
ok: [10.0.139.51] => {
“var”: {
“groups.mongod_rs_member”: [
“10.0.136.46”,
“10.0.139.51”,
“10.0.137.189”
]
}
}
ok: [10.0.137.189] => {
“var”: {
“groups.mongod_rs_member”: [
“10.0.136.46”,
“10.0.139.51”,
“10.0.137.189”
]
}
}
TASK: [mongodb | set_fact ] ***************************************************
ok: [10.0.136.46]
TASK: [mongodb | var groups.mongod_rs_member[0]] ******************************
ok: [10.0.136.46] => {
“var”: {
“groups.mongod_rs_member[0]”: “10.0.136.46”
}
}
TASK: [mongodb | shell echo “HELLO groups.mongod_rs_member +++++++++++++++++”] ***
changed: [10.0.136.46 → 10.0.136.46]
TASK: [mongodb | var mongodb_candidate_master] ********************************
ok: [10.0.136.46] => {
“var”: {
“mongodb_candidate_master”: “10.0.136.46”
}
}
TASK: [mongodb | shell echo “HELLO mongodb_candidate_master +++++++++++++++++”] ***
changed: [10.0.136.46 → 10.0.136.46]
TASK: [mongodb | var mongodb_replica_set_hosts[0]] ****************************
ok: [10.0.136.46] => {
“var”: {
“mongodb_replica_set_hosts[0]”: “10.0.136.46”
}
}
TASK: [mongodb | msg mongodb_replica_set_hosts[0]] ****************************
ok: [10.0.136.46] => {
“msg”: “10.0.136.46”
}
TASK: [mongodb | debug msg=“{”] ***********************************************
ok: [10.0.136.46] => {
“msg”: “10.0.136.46”
}
TASK: [mongodb | shell echo “HELLO mongodb_replica_set_hosts +++++++++++++++++”] ***
fatal: [10.0.136.46 → {] => SSH Error: data could not be sent to the remote host. Make sure this host can be reached over ssh
FATAL: all hosts have already failed – aborting
`
The last debug is interesting. I does not have a -name and it seems to render the value as ‘{’ just like the delegate_to.
Is it still unclear Brian?
I also seem to be seeing a related problem. Consider:
`
- name: Provision Ops RDS instance
hosts: localhost
connection: local
gather_facts: no
tags: [control]
tasks:
…
- shell: |
echo {{groups.consul_server[0]}} && hostname -i && curl localhost:8500/v1/catalog/services
register: __catalog
delegate_to: groups.consul_server[0]
`
Here the shell commands runs on localhost, not the host specified. The output is
TASK: [shell echo {{groups.consul_server[0]}} && hostname -i && curl localhost:8500/v1/catalog/services ] *** <groups.consul_server[0]> REMOTE_MODULE command echo 10.0.196.116 && hostname -i && curl localhost:8500/v1/catalog/services #USE_SHELL <groups.consul_server[0]> EXEC ['/bin/sh', '-c', 'mkdir -p $HOME/.ansible/tmp/ansible-tmp-1437044596.27-54518627543713 && chmod a+rx $HOME/.ansible/tmp/ansible-tmp-1437044596.27-54518627543713 && echo $HOME/.ansible/tmp/ansible-tmp-1437044596.27-54518627543713'] <groups.consul_server[0]> PUT /tmp/tmpTGBlM3 TO /home/bkaplan/.ansible/tmp/ansible-tmp-1437044596.27-54518627543713/command <groups.consul_server[0]> EXEC ['/bin/sh', '-c', u'LANG=en_US.UTF-8 LC_CTYPE=en_US.UTF-8 /usr/bin/python /home/bkaplan/.ansible/tmp/ansible-tmp-1437044596.27-54518627543713/command; rm -rf /home/bkaplan/.ansible/tmp/ansible-tmp-1437044596.27-54518627543713/ >/dev/null 2>&1'] failed: [localhost -> groups.consul_server[0]] => {"changed": true, "cmd": "echo 10.0.196.116 && hostname -i && curl localhost:8500/v1/catalog/services", "delta": "0:00:00.012358", "end": "2015-07-16 16:33:16.332372", "rc": 7, "start": "2015-07-16 16:33:16.320014", "warnings": []} stderr: % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to localhost port 8500: Connection refused stdout: 10.0.196.116 127.0.1.1
Where ‘10.0.196.116’ is groups.consul_server[0]. So the variable resolves correctly, but delegate to does not seem to use it. This is not specific to shell, it has the same behavior for other modules (eg, uri).
you need moustaches:
delegate_to: "{{groups.consul_server[0]}}"
only conditionals (when: ) do not need templating, with_ has a
haphazard support for it which we plan to deprecate.
Hmm, thought I tried that, but will check. Thanks Brian.
I wonder whether it would be better for it to be error if delegate_to does not resolve rather than silently fall back to localhost… Depending on the task fallback to localhost could do some serious damage.
I see the same behavior with the moustaches:
`
- name: Test delegate to
shell: echo “{{ groups.consul_server[0] }}” && hostname -i
register: __hostname
delegate_to: “{{ groups.consul_server[0] }}”
- debug: var=__hostname
`
`
TASK: [Test delegate to] *********************************************
<10.0.196.116> REMOTE_MODULE command echo “10.0.196.116” && hostname -i #USE_SHELL
<10.0.196.116> EXEC [‘/bin/sh’, ‘-c’, ‘mkdir -p $HOME/.ansible/tmp/ansible-tmp-1437108695.64-26819269533917 && chmod a+rx $HOME/.ansible/tmp/ansible-tmp-1437108695.64-26819269533917 && echo $HOME/.ansible/tmp/ansible-tmp-1437108695.64-26819269533917’]
<10.0.196.116> PUT /tmp/tmpDXWJFH TO /home/bkaplan/.ansible/tmp/ansible-tmp-1437108695.64-26819269533917/command
<10.0.196.116> EXEC [‘/bin/sh’, ‘-c’, u’LANG=en_US.UTF-8 LC_CTYPE=en_US.UTF-8 /usr/bin/python /home/bkaplan/.ansible/tmp/ansible-tmp-1437108695.64-26819269533917/command; rm -rf /home/bkaplan/.ansible/tmp/ansible-tmp-1437108695.64-26819269533917/ >/dev/null 2>&1’]
changed: [localhost → 10.0.196.116] => {“changed”: true, “cmd”: “echo "10.0.196.116" && hostname -i”, “delta”: “0:00:00.004458”, “end”: “2015-07-17 10:21:35.693136”, “rc”: 0, “start”: “2015-07-17 10:21:35.688678”, “stderr”: “”, “stdout”: “10.0.196.116\n127.0.1.1”, “warnings”: }
TASK: [debug var=__hostname] **************************************************
ok: [localhost] => {
“var”: {
“__hostname”: {
“changed”: true,
“cmd”: “echo "10.0.196.116" && hostname -i”,
“delta”: “0:00:00.004458”,
“end”: “2015-07-17 10:21:35.693136”,
“invocation”: {
“module_args”: “echo "10.0.196.116" && hostname -i”,
“module_name”: “shell”
},
“rc”: 0,
“start”: “2015-07-17 10:21:35.688678”,
“stderr”: “”,
“stdout”: “10.0.196.116\n127.0.1.1”,
“stdout_lines”: [
“10.0.196.116”,
“127.0.1.1”
],
“warnings”:
}
}
}
`
well, the first difference is that you are not delegating to a host
named 'groups.consul_server[0]' but to 10.0.196.116
Sorry for not noticing before, but the delegate_to on debug is
irrelevant, debug always executes on 'the master' or machine you
execute ansible on, it does not need to connect to any machine, i'm
not sure what you are expecting there.
delegate_to on shell or any module that executes remotely goes to the
delegated server, but things that operate on 'master' like debug,
prompt, add_host, group_by make no sense to delegate.
Brian, I’m not following you at all. It was the shell command that uses the delegate_to. The debug is only showing the register from the shell.
`
-
name: Test delegate to
shell: echo “{{ groups.consul_server[0] }}” && hostname -i
register: __hostname
delegate_to: “{{ groups.consul_server[0] }}” -
debug: var=__hostname
`
"stdout_lines": [ "10.0.196.116", <--- echo "{{ groups.consul_server[0] }}" "127.0.1.1" <--- hostname -i ],
ah, formatting of previous email threw me off, the only reason i can
think of for this happening is that you have connection=local for the
task (from play or command line) and the host does not have it
specified through ansible_connection.
All my hosts come from the ec2 inventory plugin. I can access the hosts in all other ways without having to explicitly specify an ansible_connection.
So then is it better to specify the target host in the play and use local_action? I kinda thought thesetwo were inverses of each other.
-barry
Perfect! I have removed the connection=local from the play and now all works as expected. I guess I don’t fully understand why connection=local should be used. When I was first starting with ansible plays that I studied that did ec2 stuff used it, so I did too.