# Is it intentional to authorize Python escaping in YAML playbooks? **URL:** \ **Category:** Ansible Project\ **Created:** [May 22, 2013, 1:00pm UTC](https://forum.ansible.com/t/is-it-intentional-to-authorize-python-escaping-in-yaml-playbooks/12844 "2013-05-22T13:00:45Z")\ **Posts on this page:** 15\ **Page:** 1
**Author:** ![Nicolas\_Grilly](https://avatars.discourse-cdn.com/v4/letter/n/77aa72/32.png) [@Nicolas\_Grilly](https://forum.ansible.com/u/Nicolas_Grilly)\ **Post date:** [May 22, 2013, 1:00pm UTC](https://forum.ansible.com/t/is-it-intentional-to-authorize-python-escaping-in-yaml-playbooks/12844/1 "2013-05-22T13:00:45Z")
I’ve noticed that when I use an escape sequence like “\n” in my YAML playbooks, it is interpreted by Python modules like a newline, instead of a two separated characters "" and “n”. Is it intended? It seems redundant because YAML already offers its own escaping mechanism. I understand the related code is in lib/runner/ **init**.py, in Runner.\_copy\_module: encoded\_args = “"""%s"""” % module\_args.replace(“"”,“\"”) Maybe it could be replaced by something like this: encoded\_args = module\_args.encode(‘string\_escape’) What is your opinion about this? Cheers, Nicolas ---
**Author:** ![Michael\_DeHaan2](https://avatars.discourse-cdn.com/v4/letter/m/da6949/32.png) [@Michael\_DeHaan2](https://forum.ansible.com/u/Michael_DeHaan2)\ **Post date:** [May 22, 2013, 1:28pm UTC](https://forum.ansible.com/t/is-it-intentional-to-authorize-python-escaping-in-yaml-playbooks/12844/2 "2013-05-22T13:28:43Z")
Can you please show me the example of this in a playbook and how you are observing this behavior (playbook output in verbose mode, etc)? ---
**Author:** ![Nicolas\_Grilly](https://avatars.discourse-cdn.com/v4/letter/n/77aa72/32.png) [@Nicolas\_Grilly](https://forum.ansible.com/u/Nicolas_Grilly)\ **Post date:** [May 22, 2013, 3:38pm UTC](https://forum.ansible.com/t/is-it-intentional-to-authorize-python-escaping-in-yaml-playbooks/12844/3 "2013-05-22T15:38:43Z")
Here is an example with this simple playbook: - hosts: all tasks: - copy: content=hello\nworld dest=/vagrant/testcopy.txt I run it using: $ ANSIBLE\_KEEP\_REMOTE\_FILES=1 ansible-playbook -i hosts playbook.yml And get: TASK: [copy content=hello\nworld dest=/vagrant/testcopy.txt] \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\* failed: [127.0.0.1] =\> {“failed”: true} msg: this module requires key=value arguments ([‘content=hello’, ‘world’, ‘dest=/vagrant/testcopy.txt’, ‘src=/home/vagrant/.ansible/tmp/ansible-1369233251.42-198707021399580/source’, ‘original\_basename=tmpK0KdKX’]) I do not expect it to fail. I had a look to the module uploaded to the server after replacement of MODULE\_ARGS and here is its value: MODULE\_ARGS = “”“content=hello\nworld dest=/vagrant/testcopy.txt src=/home/vagrant/.ansible/tmp/ansible-1369233251.42-198707021399580/source original\_basename=tmpK0KdKX”“” I think this is incorrect. “\n” should be escaped like this: MODULE\_ARGS = “”“content=hello\nworld dest=/vagrant/testcopy.txt src=/home/vagrant/.ansible/tmp/ansible-1369233251.42-198707021399580/source original\_basename=tmpK0KdKX”“” or like this (by using a Python raw string prefixed with a “r”): MODULE\_ARGS = r"““content=hello\nworld dest=/vagrant/testcopy.txt src=/home/vagrant/.ansible/tmp/ansible-1369233251.42-198707021399580/source original\_basename=tmpK0KdKX””" Any advice? Thanks for your help. Nicolas ---
**Author:** ![Michael\_DeHaan2](https://avatars.discourse-cdn.com/v4/letter/m/da6949/32.png) [@Michael\_DeHaan2](https://forum.ansible.com/u/Michael_DeHaan2)\ **Post date:** [May 22, 2013, 3:47pm UTC](https://forum.ansible.com/t/is-it-intentional-to-authorize-python-escaping-in-yaml-playbooks/12844/4 "2013-05-22T15:47:53Z")
Ah ok, don’t use content in this case without quoting it, and you should be fine. It’s also preferred you use “src” anyway, content was a recent addition. ---
**Author:** ![Nicolas\_Grilly1](https://avatars.discourse-cdn.com/v4/letter/n/d9b06d/32.png) [@Nicolas\_Grilly1](https://forum.ansible.com/u/Nicolas_Grilly1)\ **Post date:** [May 22, 2013, 4:35pm UTC](https://forum.ansible.com/t/is-it-intentional-to-authorize-python-escaping-in-yaml-playbooks/12844/5 "2013-05-22T16:35:06Z")
> Ah ok, don't use content in this case without quoting it, and you should > be fine. > > It's also preferred you use "src" anyway, content was a recent addition. The problem is not specific to module copy with the content arg. I used this just as an example. The problem happens with all Python modules. When args are "injected" in the Python module before being uploaded to the server, they should be escaped but are not. Here is another example with lineinfile: - hosts: all   tasks:     - copy: content='hello' dest=/vagrant/test.txt     - lineinfile: dest=/vagrant/test.txt regexp='^hello' line='hello\nworld' After running this playbook, test.txt should contain : $ cat test.txt hello\nworld Instead, I got: $ cat test.txt hello world The reason why is because "\n" is not escaped before being "injected" in MODULE\_ARGS and thus is interpreted by Python as a newline instead of just two characters "\" and "n". This behavior is undocumented, and I understand that Ansible tries hard to not be Python specific. For most use cases, one should just need to write YAML playbooks and use the built-in modules. But this small problem makes the playbook implicitly dependent on Python literal string escaping rules. What would you think of a patch trying to replace (in Runner.\_copy\_file):     encoded\_args = "\"\"\"%s\"\"\"" % module\_args.replace("\"","\\\"") by something like this:     encoded\_args = repr(module\_args) An example to illustrate the difference: $ python > > > module\_args = r'regexp="hello\nworld"' > > > print "\"\"\"%s\"\"\"" % module\_args.replace("\"","\\\"") """regexp=\"hello\nworld\"""" > > > print repr(module\_args) 'regexp="hello\\nworld"' This is the end of this long message. Sorry! ---
**Author:** ![Michael\_DeHaan2](https://avatars.discourse-cdn.com/v4/letter/m/da6949/32.png) [@Michael\_DeHaan2](https://forum.ansible.com/u/Michael_DeHaan2)\ **Post date:** [May 22, 2013, 6:34pm UTC](https://forum.ansible.com/t/is-it-intentional-to-authorize-python-escaping-in-yaml-playbooks/12844/6 "2013-05-22T18:34:59Z")
Sure, how about sending me a pull request so we can test it out and make sure it doesn’t break anything? ---
**Author:** ![Nicolas\_Grilly1](https://avatars.discourse-cdn.com/v4/letter/n/d9b06d/32.png) [@Nicolas\_Grilly1](https://forum.ansible.com/u/Nicolas_Grilly1)\ **Post date:** [May 22, 2013, 6:57pm UTC](https://forum.ansible.com/t/is-it-intentional-to-authorize-python-escaping-in-yaml-playbooks/12844/7 "2013-05-22T18:57:55Z")
Ok. I can give it a try, but I expect such a change to break some examples of lineinfile that use backslash in regexp. Would you be ok to adapt the examples if it's necessary (basically replacing double backslash by a single backslash), or would it be a show stopper? ---
**Author:** ![Michael\_DeHaan2](https://avatars.discourse-cdn.com/v4/letter/m/da6949/32.png) [@Michael\_DeHaan2](https://forum.ansible.com/u/Michael_DeHaan2)\ **Post date:** [May 23, 2013, 2:34am UTC](https://forum.ansible.com/t/is-it-intentional-to-authorize-python-escaping-in-yaml-playbooks/12844/8 "2013-05-23T02:34:22Z")
Backrefs are a pretty new feature (I believe 1.2) in lineinfile, so I’m ok with a patch that also updates them. Would be good if you can test those too if you can. Thanks! ---
**Author:** ![Nicolas\_Grilly](https://avatars.discourse-cdn.com/v4/letter/n/77aa72/32.png) [@Nicolas\_Grilly](https://forum.ansible.com/u/Nicolas_Grilly)\ **Post date:** [May 23, 2013, 11:08pm UTC](https://forum.ansible.com/t/is-it-intentional-to-authorize-python-escaping-in-yaml-playbooks/12844/9 "2013-05-23T23:08:47Z")
Done 🙂 Here is the pull request: [https://github.com/ansible/ansible/pull/3005](https://github.com/ansible/ansible/pull/3005) ---
**Author:** ![lucc](https://avatars.discourse-cdn.com/v4/letter/l/34f0e0/32.png) [@lucc](https://forum.ansible.com/u/lucc)\ **Post date:** [July 16, 2013, 7:44pm UTC](https://forum.ansible.com/t/is-it-intentional-to-authorize-python-escaping-in-yaml-playbooks/12844/10 "2013-07-16T19:44:03Z")
Hi, I would disagree with Nicolas in that \ escaping is not Python specific. Indeed, it seems to be theYAML-way to escape control characters according to YAML (see: [http://yaml.org/spec/current.html#id2517668](http://yaml.org/spec/current.html#id2517668)). It is also the natural way of regular expressions ([http://www.regular-expressions.info/reference.html](http://www.regular-expressions.info/reference.html)) The command bellow should generate line breaks and tabs, and it used to work with previous versions of ansible, both from command line or from a playbooks. tasks: - copy: content=“hello\nworld” dest=“/some/file” - lineinfile: dest=/vagrant/test.txt regexp=‘^hello’ line=‘hello\tworld’ Recently, I upgraded ansible to 1.2.2 (pip) and 1.3 (git) and now it is broken. I’ve tried other forms of escaping (\x09, \u0009), both with single and double quotes, and none have worked here. The escape sequence is always preserved in its literal form. Is there another standard way of escaping control characters that I’m missing? If not possible to revert to the previous (and correct in my PoV) behavior, I would appreciate any hints in how to do proper do escaping of control chars in string parameters. Best regards, Luciano ---
**Author:** ![Michael\_DeHaan2](https://avatars.discourse-cdn.com/v4/letter/m/da6949/32.png) [@Michael\_DeHaan2](https://forum.ansible.com/u/Michael_DeHaan2)\ **Post date:** [July 16, 2013, 9:15pm UTC](https://forum.ansible.com/t/is-it-intentional-to-authorize-python-escaping-in-yaml-playbooks/12844/11 "2013-07-16T21:15:38Z")
Doesn’t seem likely to have changed: [https://github.com/ansible/ansible/commits/devel/library/files/lineinfile](https://github.com/ansible/ansible/commits/devel/library/files/lineinfile) I suspect you were using something earlier than 1.2.2 before maybe, back when it didn’t do backrefs. ---
**Author:** ![lucc](https://avatars.discourse-cdn.com/v4/letter/l/34f0e0/32.png) [@lucc](https://forum.ansible.com/u/lucc)\ **Post date:** [July 17, 2013, 3:00pm UTC](https://forum.ansible.com/t/is-it-intentional-to-authorize-python-escaping-in-yaml-playbooks/12844/12 "2013-07-17T15:00:11Z")
Hi, I think I’ve misread YAML specifications. Actually it only does expand escape sequences (\c, \x\*\*, \u\*\*\*\*) for **double quoted scalar blocks**. So, while using double-quotes in the example bellow, actually, from PoV of YAML’s parser it is a _plain scalar block_, because it does not start with double-quotes. tasks: copy: content=“beforetab\x09after” dest=“/some/file” I’ve found two solutions for the problem: 1. Convert value to double-quote scalar block (must escape any inner double-quotes in the block or replace by single-quotes): tasks: copy: “content=‘beforetab\x09after’ dest=‘/some/file’” 1. Express copy parameters as a dict and use double-quotes for each value that need to use escape sequences. tasks: copy: content: “beforetab\x09after” dest: /some/file However, I’m not sure solution 2 is officially supported or is a hidden feature of ansible. I was not able to find this information in documentation currently available for playbooks. BTW, CLI is also working. I forgot expansion of escape sequences does not occur depending on the used unix shell. So in bash, for instance, one need to Ctrl+V to insert tab in string, instead of using \t. My mistake. Best regards, Luciano ---
**Author:** ![Michael\_DeHaan2](https://avatars.discourse-cdn.com/v4/letter/m/da6949/32.png) [@Michael\_DeHaan2](https://forum.ansible.com/u/Michael_DeHaan2)\ **Post date:** [July 17, 2013, 4:49pm UTC](https://forum.ansible.com/t/is-it-intentional-to-authorize-python-escaping-in-yaml-playbooks/12844/13 "2013-07-17T16:49:03Z")
Best to just transfer a file and not use “content” IMHO. ---
**Author:** ![lucc](https://avatars.discourse-cdn.com/v4/letter/l/34f0e0/32.png) [@lucc](https://forum.ansible.com/u/lucc)\ **Post date:** [July 17, 2013, 5:36pm UTC](https://forum.ansible.com/t/is-it-intentional-to-authorize-python-escaping-in-yaml-playbooks/12844/14 "2013-07-17T17:36:09Z")
I would agree, except when you need to generate pretty small and somewhat formatted content on-the-fly. Ex.: passing some form of credentials to remote node… Anyway, I’m using lineinfile. I just used ‘copy’ in the example because I thought it would became simpler to explain the issue with YAML. Thanks for your help, s ---
**Author:** ![Nicolas\_Grilly](https://avatars.discourse-cdn.com/v4/letter/n/77aa72/32.png) [@Nicolas\_Grilly](https://forum.ansible.com/u/Nicolas_Grilly)\ **Post date:** [July 17, 2013, 7:03pm UTC](https://forum.ansible.com/t/is-it-intentional-to-authorize-python-escaping-in-yaml-playbooks/12844/15 "2013-07-17T19:03:28Z")
Hi,