passing data to roles via parameters without accidentally flattening it

Archives Ansible Project
1

Hello,

I’ve been having some problems passing parameters to roles. Essentially the data in my parameter is getting flattened into a string using {{}} interpolation, but not with ${} interpolation. Is there a recommended way to pass parameters like this? I want top be able to define lists of dictionaries. pass them to roles, and have the roles loop over them.

A toy example - given a playbook like this which passes a list parameter ‘databases’ to a role called ‘mysql’:

---
- name: test roles
hosts: all
user: root
roles:
- role: mysql
databases:
```- ‘{{my.mysql.database}}’`

The mysql role consists only of this tasks/main.yml file:

---
- debug: msg="database is '{{databases[0].name}}'"

- debug: msg="database is '{{item}}'"
with_items: '{{databases}}'

I am defining my.mysql.database variable in host_vars/all:

my:
mysql:
database:
name: fred

I get an error implying that the parameter’s data has been flattened into a string, as follows:

TASK: [debug msg="database is '{{databases[0].name}}'"] ***********************
fatal: [lamian.noodlefactory.co.uk] => One or more undefined variables: 'unicode object' has no attribute 'name'

Removing the quotes is no good, it gets a YAML syntax error.

ERROR: Syntax Error while loading YAML script, test-roles.yml
Note: The error may actually appear before this position: line 8, column 12

databases:
- {{my.drupal.database}}
^

But what does work is using the old dollar syntax, ${my.mysql.database}, with no quotes:

TASK: [debug msg="database is '{{databases[0].name}}'"] ***********************
ok: [lamian.noodlefactory.co.uk] => {"item": "", "msg": "database is 'fred'"}

TASK: [debug msg="database is ''"] ********************************************
ok: [lamian.noodlefactory.co.uk] => (item={'name': 'fred'}) => {"item": {"name": "fred"}, "msg": "database is '{'name': 'fred'}'"}

However, I gather that syntax is discouraged these days. Is there a way of avoiding it in this case?

Cheers,

Nick

2

Whoops. This should actually read {{my.mysql.database}} to be consistent with my
example. But the point stands.

3

Good timing on this post. This looks like the same issue I’ve been dealing with today. Does it work if you load the variables using vars_files instead of in host_vars?

Nathan

4

If you are referencing variables inside other datastructures this is the one case where you still have to do $foo.

We're currently exploring how to eliminate this.

5

Excerpts from Michael DeHaan's message of 2013-09-06 08:33:06 -0400:

If you are referencing variables inside other datastructures this is the
one case where you still have to do $foo.

Having run into this also, I have a couple cents regarding it and some
other difficulties I've run into with variables also.

I think it's a really *nice* feature that jinja2-style variables in
playbooks work almost identically to real jinja2 templates. That is to
say, the current functionality can be reasoned about as though your
playbooks are simply being run through jinja2 as templates. The fact
that '{{ foo.fanciness|whatever_magic_you_want is defined }}' *always*
evaluates to a string is, in my opinion, awesome.

If you can avoid tromping on that, it would be awesome. That said, the
complex-arguments-to-modules use case and the references use case are
really compelling, and I'd love a solution there too.

Perhaps related to the references use case is that the following still
works in, say, a vars file:

    foo:
      bar: hello
      nar: ${foo.hello}/noway/howdy

But this doesn't:

    foo:
      bar: hello
      nar: "{{ foo.hello }}/noway/howdy"

While it's reasonable that it doesn't work, considering the
self-reference, it's also really nice to be able to do sometimes, and
allows the use of hash tables as role variable namespaces instead of the
'role_' prefix that some folks are using.

6

role variable namespaces are pretty unnecessary in 1.3 due to the way you are always guaranteed to get the value from the role in that current role.

7

Excerpts from Michael DeHaan's message of 2013-09-06 08:57:07 -0400:

role variable namespaces are pretty unnecessary in 1.3 due to the way you
are always guaranteed to get the value from the role in that current role.

Oh jeez, I completely missed that implication of the feature!

Thanks!

8

​The problem here is you need to quote it:​

9

I confess I've not tried this, since Michael says the dollar is required.

N

10

Hi,

11

Hi,

I've been converting more of my older playbooks to the new brace expansion syntax, I've found another case where dollar expansion succeeds to do something which braces don't: variable names with leading numerals. In my case a variable was called 'something.3rd_party_rpms', and converting to the new syntax made a working playbook fail.

I'm not clear if this is a bug, rather I suspect it's a consequence of Jinja2's Pythonesque syntax, which I think disallows leading numeral identifiers for variable names. However, documentation on this would benefit those of us converting from the older syntax. If someone who knows can confirm whether or not these cases are how it is meant to be, given a spare moment I may provide a documentation patch.

In addition I notice that error_on_undefined_vars does not spot these defined-but-unexpanded vars with leading numerals.

And as I mention elsewhere, checking this shows another surprising difference, which is that undefined dollar expansions don't seem to be caught at all when error_on_undefined_vars = True.

Here's a test playbook:

---# test.yml
- name: test moustache vs dollar expansion
   hosts: all
   user: root

   vars:
     foo1: /foo1
     foo:
       bar: /foo/bar
       3bar: /foo/3bar

   tasks:
     - name: what is foo1?
       action: debug msg="{{foo1}} ${foo1}"
       # prints "/foo1 /foo1"

     - name: what is foo.bar?
       action: debug msg="{{foo.bar}} ${foo.bar}"
       # prints "/foo/bar /foo/bar"

     - name: what is foo.3bar?
       action: debug msg="{{foo.3bar}} ${foo.3bar}"
       # prints "{{foo.3bar}} /foo/bar",
       # even if error_on_undefined_vars = True!

     - name: what is foo.nonesuch?
       action: debug msg="${foo.nonesuch}"
       # prints "${foo.nonesuch}",
       # even if error_on_undefined_vars = True!

     - name: what is foo.nonesuch?
       action: debug msg="{{foo.nonesuch}}"
       # this does trigger an error if error_on_undefined_vars = True!

Run like this:

     ansible-playbook -l localhost -c local -v test.yml

Gives:

GATHERING FACTS

12

You should never define a variable name with a leading numeral, as it’s not a valid Python variable name.

13

most of us never try this as most programming languages don’t like variables that way.

14

Just to clarify: is the following referencing variables inside other datastructures?

--- # vars.yml
foo:
   base: /some/path
path: '{{ foo.base }}/filename'

(Works as expected to set path to "/some/path/filename")

How about this?

--- # vars.yml
foo:
   base: /some/path
   path: '{{ foo.base }}/filename'

(Ansible gives an error: "recursive loop detected in template string: {{foo.base}}/filename".)

How about this?

--- # vars.yml
foo:
   base: /some/path
   path: '${foo.base}/filename'

(Works as expected to set path to "/some/path/filename")

My point is, when dollar variables are no longer supported in Ansible, will we be unable to express the second case?

N

15

I do know that about Python variables. I also know that these are valid YAML identifiers, and valid Python dictionary keys too.

So it's not clear-cut which rules apply to Ansible vars, which behave a lot like keys in a nested dictionary. Ansible's docs should say but don't seem to - or perhaps I just haven't found that bit.

I'm suggesting Ansible should:

  - Document what is allowed, and what is advised, in terms of variable names, especially when the rules change (as in this case).

  - Warn the user when he or she breaks the rules / advice (currently, whether I find out is hapazard).

  - Implement error_on_undefined_vars checks universally, or document the exceptions (currently the loophole with leading numerals, and dollar vars in general)

These do seem pretty uncontroversial points, I hope. But want to know if you agree, or these are not going to change because of a specific policy, so that I can do the right thing.

Cheers,

N

16

Hi,

I am not sur this is related… but the symptoms are the same.

I have a variable set this way:

asadmin_cmd: "/path/to/glassfish/bin/asadmin --user admin --passwordfile /path/to/passwordfile"

in my playbook I am trying the following:

        - name: disable X-Powered-By
          shell: "{{ asadmin_cmd }}" set server.network-config.protocols.protocol.http-listener-1.http.xpowered-by=false

which return the following error message:

ERROR: Syntax Error while loading YAML script, ….

If I try :

        - name: disable X-Powered-By
          shell: ${asadmin_cmd} set server.network-config.protocols.protocol.http-listener-1.http.xpowered-by=false

everything runs smoothly.

Fred

17

The error looks like it from the YAML parser, implying Ansible hasn't even had a
chance of interpreting the data yet.

I believe it's the quotes. If I paste your definitions into a test playbook and
put quotes around "${ asadmin_cmd }" I get the same sort of YAML error. YAML
apparently wants the quotes to enclose the whole value, and not just a portion.
Quoting the whole value works:

shell: "{{ asadmin_cmd }} set server.network-config.protocols.protocol.http-listener-1.http.xpowered-by=false"

N

18

Yep, this is just a parser issue.

If you start a line with a quote, you have to make sure that quote applies to the whole line.

I agree this is a little confusing.

In previous thread we discussed including some built-in YAML parser error detection that would better explain this, and how to fix it, and on what line numbers the errors were so you didn’t see a traceback.

Something I hope we can add in the next release or two.

19

I should mention what you are doing appears to be because you’re trying to namespace variables.

In Ansible 1.3,you no longer have to, because it’s guaranteed any variable you have defined in a role comes will be used in that role.

Thus it’s easier to just define everything flat if you want.

I agree we should document that variables shouldn’t contain dashes.

I’m not sure what you mean about error_on_undefined not picking up everything, but I certaintly believe that it’s possible that it’s not 100% yet, patches or examples would be great – please file specific tickets in github about what error_on_undefined is not picking up so we don’t miss this.

20

Hi Nick, Mike,

Thank you for the tip, quoting the all line does indeed suppress the error message.
I suspected I was missing some subtleties here :slight_smile:

I was referring to the http://www.ansibleworks.com/docs/YAMLSyntax.html#gotchas page which explain that a value after a colon that start with { needs to be quoted … but did not get that the all line was concerned.

Thanks again for your help.

fred