vbotka
(Vladimir Botka)
March 6, 2025, 10:51am
1
The below expressions are valid YAML dictionaries
{"socket": "test"}
{socket: "test"}
They become valid YAML strings if you quote them
'{"socket": "test"}'
'{socket: "test"}'
When you put a substitution into the expression, given the variable
test_var: test
the below expression evaluates to a string
'{socket: "{{ test_var }}"}'
, but if the ‘key’ is quoted
'{"socket": "{{ test_var }}"}'
it evaluates to a dictionary
The evaluation becomes weirder when you append a comma. The string remains a string
'{socket: "{{ test_var }}"},'
, but the dictionary becomes a list with a single item [{'socket': 'test'}]
'{"socket": "{{ test_var }}"},'
In addition to this. If you write it to a file
- copy:
dest: /tmp/test.txt
content: |
{"socket": "{{test_var}}"},
you get a set
shell> cat /tmp/test.txt
({'socket': 'test'},)
Is this a bug?
If it isn’t a bug can you point me to a docs that explain this?
Example of a complete playbook for testing
- hosts: localhost
vars:
test_var: test
tasks:
- debug:
msg: |
x: {{ item }}
type(x): {{ item | type_debug }}
loop:
- {"socket": "test"}
- {socket: "test"}
- '{"socket": "test"}'
- '{socket: "test"}'
- '{"socket": "{{ test_var }}"}'
- '{socket: "{{ test_var }}"}'
- '{"socket": "{{ test_var }}"},'
- '{socket: "{{ test_var }}"},'
1 Like
sivel
(sivel)
March 6, 2025, 2:59pm
2
This isn’t a bug, it’s just a side effect of how templating works, and the implementation historically necessary to get jinja2 to be able to provide python data types, instead of just strings. See:
opened 07:49PM - 08 Jan 18 UTC
closed 08:46PM - 08 Jan 18 UTC
python3
module
support:core
affects_2.5
bug
##### ISSUE TYPE
- Bug Report
##### COMPONENT NAME
copy_module
##### AN… SIBLE VERSION
```
ansible 2.5.0 (devel 34206a0402) last updated 2018/01/08 18:52:56 (GMT +000)
config file = None
configured module search path = ['/home/ubuntu/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /home/ubuntu/ansible/lib/ansible
executable location = /home/ubuntu/ansible/bin/ansible
python version = 3.5.2 (default, Nov 23 2017, 16:37:01) [GCC 5.4.0 20160609]
```
also seen on:
```
ansible 2.4.2.0
config file = /etc/ansible/ansible.cfg
configured module search path = ['/home/brian/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /home/brian/.virtualenvs/mypy/lib/python3.5/site-packages/ansible
executable location = /home/brian/.virtualenvs/mypy/bin/ansible
python version = 3.5.2 (default, Nov 17 2016, 17:05:23) [GCC 5.4.0 20160609]
```
##### CONFIGURATION
```
(ansible-test) ubuntu@ubuntu-xenial:~/ansible$ ansible-config dump --only-changed
(ansible-test) ubuntu@ubuntu-xenial:~/ansible$
```
(no output -- no changes to ansible.cfg, this is a clean ubuntu 16.04 VM with python, pip, virtualenv, a clone of the ansible repo, and `pip -r requirements.txt`)
##### OS / ENVIRONMENT
ubuntu 16.04, managing itself
##### SUMMARY
When running under python 3, using copy.content with literal json and a variable in the input, the output varies from run to run (the dict keys are being reordered arbitrarily).
##### STEPS TO REPRODUCE
Run the playbook below with:
```
ansible-playbook -i localhost, playbook.yaml
```
```yaml
---
- hosts: localhost
connection: local
gather_facts: false
vars:
my_var: a_value
tasks:
- copy:
dest: ./output.txt
content: |
{
"key1": "{{ my_var }}",
"key2": "something else",
"key3": "something different",
"key4": "completely different",
"key5": "1.2.3"
}
```
Note in the example playbook:
1. This only seems to happen when using a variable (static content didn't show this behavior).
2. This only seems to happen when the input is json (yaml didn't show this behavior).
##### EXPECTED RESULTS
1. The provided input should be output in the same format (i.e. without having whitespace rearranged).
2. The provided input should not be reordered (i.e. dict keys should not "move around").
3. (As a corollary to 2) The playbook should yield changed=0 on the second run -- the output file should not be modified.
##### ACTUAL RESULTS
See example run below.
Note that:
1. The input has been reformatted onto a single line.
2. The keys are reordered from the input.
3. The file changes on every run.
```
(ansible-test) ubuntu@ubuntu-xenial:~/ansible$ ansible-playbook -i localhost, playbook.yaml
PLAY [localhost] *******************************************************************************************************************************************
TASK [copy] ************************************************************************************************************************************************
changed: [localhost]
PLAY RECAP *************************************************************************************************************************************************
localhost : ok=1 changed=1 unreachable=0 failed=0
(ansible-test) ubuntu@ubuntu-xenial:~/ansible$ cat output.txt
{"key1": "a_value", "key2": "something else", "key5": "1.2.3", "key3": "something different", "key4": "completely different"}(ansible-test) ubuntu@ubuntu-xenial:~/ansible$
(ansible-test) ubuntu@ubuntu-xenial:~/ansible$
(ansible-test) ubuntu@ubuntu-xenial:~/ansible$ ansible-playbook -i localhost, playbook.yaml
PLAY [localhost] *******************************************************************************************************************************************
TASK [copy] ************************************************************************************************************************************************
changed: [localhost]
PLAY RECAP *************************************************************************************************************************************************
localhost : ok=1 changed=1 unreachable=0 failed=0
(ansible-test) ubuntu@ubuntu-xenial:~/ansible$ cat output.txt
{"key5": "1.2.3", "key3": "something different", "key4": "completely different", "key1": "a_value", "key2": "something else"}(ansible-test) ubuntu@ubuntu-xenial:~/ansible$
(ansible-test) ubuntu@ubuntu-xenial:~/ansible$
(ansible-test) ubuntu@ubuntu-xenial:~/ansible$ ansible-playbook -i localhost, playbook.yaml
PLAY [localhost] *******************************************************************************************************************************************
TASK [copy] ************************************************************************************************************************************************
changed: [localhost]
PLAY RECAP *************************************************************************************************************************************************
localhost : ok=1 changed=1 unreachable=0 failed=0
(ansible-test) ubuntu@ubuntu-xenial:~/ansible$ cat output.txt
{"key1": "a_value", "key5": "1.2.3", "key4": "completely different", "key3": "something different", "key2": "something else"}(ansible-test) ubuntu@ubuntu-xenial:~/ansible$
(ansible-test) ubuntu@ubuntu-xenial:~/ansible$
```
2 Likes
vbotka
(Vladimir Botka)
March 7, 2025, 8:26am
3
For the record.
Jinja2 evaluates a substitution in an expression. Ansible does some magic to try and convert strings that look like Python data structures to Python data structures
. This undocumented behavior of a substitution may cause unpredicted malfunctions. For example, the below task
- debug:
msg: "{{ s }} {{ s | type_debug }}"
vars:
s: '{"foo"},'
gives
msg: '{"foo"}, AnsibleUnicode'
The substitution
x: foo
and the task
- debug:
msg: "{{ s }} {{ s | type_debug }}"
vars:
x: foo
s: '{"{{ x }}"},'
gives
msg: ({'foo'},) tuple
Notes:
Module copy vs. module template
- copy:
dest: /tmp/test.txt
content: |
{"{{ x }}"},
vars:
x: foo
gives
shell> cat /tmp/test.txt
({'foo'},)
In the option content , the docs says:
… if content contains a variable, use the ansible.builtin.template module.
shell> cat test.txt.j2
{"{{ x }}"},
- template:
src: test.txt.j2
dest: /tmp/test.txt
vars:
x: foo
gives
shell> cat /tmp/test.txt
{"foo"},
1 Like
vbotka
(Vladimir Botka)
March 7, 2025, 9:25am
4
Thank you. I think, this should be documented. In Manipulating data perhaps?
2 Likes
utoddl
(Todd Lewis)
March 10, 2025, 12:51am
5
I’ve always been bothered by this admonition. There are plenty of instances where the result of using content:
containing variables is exactly what’s desired.
It would be better I think to describe the difference and dangers between real templates and sneaking templated text into content:
values, or at least point to a discussion of same. (Previous messages in this topic would be a good start.) Then the user can make an informed decision.
Not that it’s bad advice. It isn’t. But without further discussion it’s like, “Here be dragons!” on old maps. “Pray, tell us more about these dragons…”
2 Likes
vbotka
(Vladimir Botka)
March 10, 2025, 8:06am
6
Technically, there is no reason why copy.content
doesn’t implement the same functionality template.src
does.
The problem isn’t the copy action, but what ansible does with content
before passing it to the copy action. With Data Tagging we’re finally a big step closer to getting rid of the need to use the template
module for some inputs.
You should really try out the Data Tagging preview: Data Tagging preview and testing