# looping over group\_vars in a task include results in mis-interpolation of $item

**URL:** <https://forum.ansible.com/t/looping-over-group-vars-in-a-task-include-results-in-mis-interpolation-of-item/12424>\
**Category:** Ansible Project\
**Created:** [February 26, 2013, 12:20am UTC](https://forum.ansible.com/t/looping-over-group-vars-in-a-task-include-results-in-mis-interpolation-of-item/12424 "2013-02-26T00:20:32Z")\
**Posts on this page:** 19\
**Page:** 1

<div class="post-metadata">

**Author:** ![Nick](https://avatars.discourse-cdn.com/v4/letter/n/f9ae1b/32.png) [@Nick](https://forum.ansible.com/u/Nick)\
**Post date:** [February 26, 2013, 12:20am UTC](https://forum.ansible.com/t/looping-over-group-vars-in-a-task-include-results-in-mis-interpolation-of-item/12424/1 "2013-02-26T00:20:32Z")

</div>

Hi,

I’m trying to manage user creation with Ansible, but I am having problems getting variable interpolation, task includes and with\_items to play nicely together. This is using a recent devel checkout, but it also seems to be a problem in 1.0 and 0.9.

Below is a stripped down example, in which users just have the pubkey attribute defined. The key idea is that a task file create.yml contains a list of tasks which should be executed for each user, and that the main playbook includes that from within a loop and sets the user attributes differently each time round.

```auto
  tasks:
    - include: create_user.yml user="${item}" pubkey="${userdefs.${item}.pubkey}"
      with_items: ${users}

```

I’ve put a fuller example in a gist here, with two versions of create\_user.yml trying different variations of the above: [https://gist.github.com/wu-lee/4994831](https://gist.github.com/wu-lee/4994831). Output of running it is appended.

The problem I find is, when the variables are defined via `group_vars/standard_users`, they don’t seem to be interpolated as you’d expect - ${item} becomes “user1,user2”, and not “user1” then “user2” in sequence. This is frustrating since I think the pattern I’m using in createA.yml would be very handy in various other circumstances, when a number of tasks need to be grouped and repeated.

Furthermore, when `${users}` is defined in a `vars:` section within the `users.yml` playbook, they do result in $item expanding to “user1”, “user2” in sequence. Some subtle interaction between variables, parameterised tasks, and `with_items`, it seems? However, I don’t want to have to use a `vars:` section because it means I can’t define a different user list by host/group via host\_vars/group\_vars.

So although it seems like it _ought_ to work, I can’t get it to. Is this a bug?

And evidently other people must do it differently, since I can’t find much evidence anyone else has this problem. Is there any prior art for creating varying sets of users I can emulate? (For now I’ve worked around it by iterating each task in create\_user.yml directly in the parent playbook, but it’s not very DRY).

One final point: in cases when a playbook actually fails to interpolate some variables (rather than interpolating them as something odd), Ansible blindly attempts to continue and, say, create users, only failing because the resulting user name is invalid. Wouldn’t it be better if Ansible refused to execute such commands with failed interpolations? I can’t imagine a situation when you’d actually _want_ Ansible to excecute commands containing the unexpanded string “${item}” (for example). On the contrary, I imagine in some circumstances it might be harmful - if the command executes successfully and does something unexpected. (This sort of non-strict behaviour in Puppet has burnt me in the past.)

Cheers,

Nick

---

<div class="post-metadata">

**Author:** ![Michael\_DeHaan](https://avatars.discourse-cdn.com/v4/letter/m/c4cdca/32.png) [@Michael\_DeHaan](https://forum.ansible.com/u/Michael_DeHaan)\
**Post date:** [February 26, 2013, 12:39am UTC](https://forum.ansible.com/t/looping-over-group-vars-in-a-task-include-results-in-mis-interpolation-of-item/12424/2 "2013-02-26T00:39:17Z")

</div>

> So although it seems like it ought to work, I can't get it to. Is this a  
> bug?

If you can reduce this down to the bare minimum possible gist, I'll take a look.

Right now, I'm failing to see the "include:" line in your gist.

BTW, the only\_if stuff can go.

"when\_set: $foo" is great.

---

<div class="post-metadata">

**Author:** ![Nick](https://avatars.discourse-cdn.com/v4/letter/n/f9ae1b/32.png) [@Nick](https://forum.ansible.com/u/Nick)\
**Post date:** [February 26, 2013, 10:07am UTC](https://forum.ansible.com/t/looping-over-group-vars-in-a-task-include-results-in-mis-interpolation-of-item/12424/3 "2013-02-26T10:07:38Z")

</div>

That is approximately the bare minimum \*working\* gist. Part of the problem is that if I put everything in one self-contained playbook it works, but if I use group\_vars, it doesn't.

If you just want to get an idea of the kind of thing I am trying to achieve, see below.

Cheers,

N

--- # users.yml  
- name: Create users  
user: root  
hosts: all  
vars: users: # This would be in group\_vars - alice - bob userdefs: # This would be a shared list alice: pubkey: sdfjwkfuqwerw # More attributes here.... bob: pubkey: ajdfhUHFWJWRWE  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;# More attributes here....  
&nbsp;&nbsp;tasks:  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;- include: create\_userA.yml user="${item}" pubkey="${userdefs.${item}.pubkey}"  
with\_items: ${users} --- # create\_userA.yml tasks:

- name: Create the "${user}" user  
action: user  
state=present  
name="${user}"  
createhome=yes  
- name: Insert authorized\_keys for user "$user"  
action: authorized\_key  
state=present  
user="${user}"  
key="${pubkey}"

---

<div class="post-metadata">

**Author:** ![Nick](https://avatars.discourse-cdn.com/v4/letter/n/f9ae1b/32.png) [@Nick](https://forum.ansible.com/u/Nick)\
**Post date:** [February 26, 2013, 10:14am UTC](https://forum.ansible.com/t/looping-over-group-vars-in-a-task-include-results-in-mis-interpolation-of-item/12424/4 "2013-02-26T10:14:20Z")

</div>

Gah, sorry, something unhelpfully mangled my indentation there.

Try this: [https://gist.github.com/wu-lee/5037444](https://gist.github.com/wu-lee/5037444)

---

<div class="post-metadata">

**Author:** ![Daniel\_Hokka\_Zakriss](https://avatars.discourse-cdn.com/v4/letter/d/f05b48/32.png) [@Daniel\_Hokka\_Zakriss](https://forum.ansible.com/u/Daniel_Hokka_Zakriss)\
**Post date:** [February 26, 2013, 10:21am UTC](https://forum.ansible.com/t/looping-over-group-vars-in-a-task-include-results-in-mis-interpolation-of-item/12424/5 "2013-02-26T10:21:10Z")

</div>

Nick wrote:

> Hi,
> 
> I'm trying to manage user creation with Ansible, but I am having problems  
> getting variable interpolation, task includes and with\_items to play  
> nicely  
> together. This is using a recent devel checkout, but it also seems to be a  
> problem in 1.0 and 0.9.
> 
> Below is a stripped down example, in which users just have the pubkey  
> attribute  
> defined. The key idea is that a task file create.yml contains a list of  
> tasks  
> which should be executed for each user, and that the main playbook  
> includes that  
> from within a loop and sets the user attributes differently each time  
> round.

This is due to how includes are processed. Because they are handled as a  
pre-processing directive, setting things as the files are loaded, you can  
only use global variables (e.g. vars: or vars\_files: without host-specific  
variables in them) for with\_items on include:.

Daniel

---

<div class="post-metadata">

**Author:** ![Nick](https://avatars.discourse-cdn.com/v4/letter/n/f9ae1b/32.png) [@Nick](https://forum.ansible.com/u/Nick)\
**Post date:** [February 26, 2013, 1:13pm UTC](https://forum.ansible.com/t/looping-over-group-vars-in-a-task-include-results-in-mis-interpolation-of-item/12424/6 "2013-02-26T13:13:37Z")

</div>

Thanks, that makes sense, but is a bit of a shame because using an include inside a loop like this seems such a useful thing to do.

Defining a fixed pool of users and creating a subset on each host/group must be a fairly common use case. Is there a nice way to do it which doesn't run into this limitation I did? I can't see anything very relevant in the examples/best practices docs so far, but it's hard to be sure without combing through every singe file. (I know about LDAP and similar non-Ansible solutions but they're not really appropriate for me.)

- N

ps

I note that my original example tries two ways of doing the creation - and something odd is going on in the second example. create\_userB.yml doesn't use parameterised variables, it just uses $item directly in the task file. The output (pasted from my first email) includes this output from a debug action:

TASK: [debug msg="user is '${users}' pubkey is '${userdefs.${item}.pubkey}'"] \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*  
ok: [some.host.net] =\> (item=${users}) =\> {"item": "${users}", "msg": "user is 'alice,bob' pubkey is '${userdefs.${item}.pubkey}'"}

I find it hard to explain why I can see "alice" and "bob" in there if ${users} isn't even in scope, but I'm guessing this is double-expansion in the debug task's message, where the second time the message is expanded ${users} is in scope and so expands to "alice,bob"?

---

<div class="post-metadata">

**Author:** ![Michael\_DeHaan](https://avatars.discourse-cdn.com/v4/letter/m/c4cdca/32.png) [@Michael\_DeHaan](https://forum.ansible.com/u/Michael_DeHaan)\
**Post date:** [February 26, 2013, 2:24pm UTC](https://forum.ansible.com/t/looping-over-group-vars-in-a-task-include-results-in-mis-interpolation-of-item/12424/7 "2013-02-26T14:24:45Z")

</div>

I don't have time to help debug a full playbook stack when you did not  
include the complete files. Gist also makes it hard to tell your  
structure. I think what you need to do is host a complete \*repo\* that  
illustrates what you are talking about, and  
the command you are running that will cause whatever.

Meanwhile, you asked a question about includes and variable scope.

Variables in ansible are late-binding, so you can reference variables  
that don't yet expand, but they can't be part of include paths and so  
on. If you use one of these variables in a task name, the name won't  
be all expanded out, but the action line  
will be totally fine.

Think of the task object as something that applies to all hosts in the  
group, and then each host decides whether or not to run them, and how  
to interpret the variables.

This results in each hosts all having the same total counts of  
operations they get to look at, which results in predictable counts as  
to how many things they have either skipped (in case of conditionally  
decided to run) or executed (pass or failed).

---

<div class="post-metadata">

**Author:** ![Nick](https://avatars.discourse-cdn.com/v4/letter/n/f9ae1b/32.png) [@Nick](https://forum.ansible.com/u/Nick)\
**Post date:** [February 26, 2013, 5:14pm UTC](https://forum.ansible.com/t/looping-over-group-vars-in-a-task-include-results-in-mis-interpolation-of-item/12424/8 "2013-02-26T17:14:17Z")

</div>

I think Daniel HZ already answered my question "why doesn't this work?", and the answer comes with the implication that this is a feature and not a bug.

For a complete repo, see below, but I don't want you to waste your time trying to debug my playbook stack. For the moment let's assume this trick can't be made to work the way things are. Given that, I would prefer to ask:

&nbsp;&nbsp;- what is a good recipe for creating, on each host/group, a subset of a site-wide predefined list of users?

Which is what I actually want to achieve, and it might be useful for many other readers.

&nbsp;&nbsp;- what is a good way to loop over a list of N tasks?

This is the more generalised question. I suspect your answer will be "you can't" but maybe there is another way?

Cheers

N

ps. A repo version of that Gist:

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[https://github.com/wu-lee/ansible-users-usecase](https://github.com/wu-lee/ansible-users-usecase)

---

<div class="post-metadata">

**Author:** ![Michael\_DeHaan](https://avatars.discourse-cdn.com/v4/letter/m/c4cdca/32.png) [@Michael\_DeHaan](https://forum.ansible.com/u/Michael_DeHaan)\
**Post date:** [February 26, 2013, 5:34pm UTC](https://forum.ansible.com/t/looping-over-group-vars-in-a-task-include-results-in-mis-interpolation-of-item/12424/9 "2013-02-26T17:34:18Z")

</div>

> > I don't have time to help debug a full playbook stack when you did not  
> > include the complete files. Gist also makes it hard to tell your  
> > structure. I think what you need to do is host a complete \*repo\* that  
> > illustrates what you are talking about, and  
> > the command you are running that will cause whatever.
> 
> I think Daniel HZ already answered my question "why doesn't this work?", and  
> the answer comes with the implication that this is a feature and not a bug.

His implication is wrong, it's a bug 🙂

There was a time when with\_items made task objects, but there is no  
reason for this anymore.  
In fact, conditional vars\_files works fine when based on facts. ex:

vars\_files:  
&nbsp;&nbsp;&nbsp;&nbsp;- ["/tmp/$ansible\_domain.yml", "/tmp/default"]

And that expands fine.

vars\_files is fine for instance, so it's just an issue with group\_vars  
inventory not showing up in this case.

> - what is a good recipe for creating, on each host/group, a subset of a  
> site-wide predefined list of users?

That with\_items doesn't expand based on inventory scope is something I  
need to fix, and I'll file a bug on this.

> - what is a good way to loop over a list of N tasks?

with\_items is still that answer.

---

<div class="post-metadata">

**Author:** ![Michael\_DeHaan](https://avatars.discourse-cdn.com/v4/letter/m/c4cdca/32.png) [@Michael\_DeHaan](https://forum.ansible.com/u/Michael_DeHaan)\
**Post date:** [February 26, 2013, 5:37pm UTC](https://forum.ansible.com/t/looping-over-group-vars-in-a-task-include-results-in-mis-interpolation-of-item/12424/10 "2013-02-26T17:37:05Z")

</div>

Ticket here:

[https://github.com/ansible/ansible/issues/2203](https://github.com/ansible/ansible/issues/2203)

---

<div class="post-metadata">

**Author:** ![Michael\_DeHaan](https://avatars.discourse-cdn.com/v4/letter/m/c4cdca/32.png) [@Michael\_DeHaan](https://forum.ansible.com/u/Michael_DeHaan)\
**Post date:** [February 26, 2013, 6:11pm UTC](https://forum.ansible.com/t/looping-over-group-vars-in-a-task-include-results-in-mis-interpolation-of-item/12424/11 "2013-02-26T18:11:16Z")

</div>

Turns out my ticket is incorrectly filed, as my test was wrong.

What you can do is define your list in groupvars/\<groupname\> or  
hostvars/\<hostname\>

and use with\_items: ${alist}

and it will do what you want.

---

<div class="post-metadata">

**Author:** ![Nick](https://avatars.discourse-cdn.com/v4/letter/n/f9ae1b/32.png) [@Nick](https://forum.ansible.com/u/Nick)\
**Post date:** [February 26, 2013, 6:43pm UTC](https://forum.ansible.com/t/looping-over-group-vars-in-a-task-include-results-in-mis-interpolation-of-item/12424/12 "2013-02-26T18:43:42Z")

</div>

Was going to just say "thanks", and that still stands, but now I'm confused.

> Turns out my ticket is incorrectly filed, as my test was wrong.
> 
> What you can do is define your list in groupvars/\<groupname\> or  
> hostvars/\<hostname\>
> 
> and use with\_items: ${alist}
> 
> and it will do what you want.

In the ticket, dhozac says:

> The only case where it doesn't work is for include, which iterates the list and duplicates the tasks/plays for each item in the list

Which is exactly what I was trying to do: bundle some tasks in an include, and repeat it using with\_items, expanding $item differently each time. Sounds like Daniel says it won't work, but you (maybe) are saying it will. Can you clarify?

&nbsp;&nbsp;(We can take this to the ticket comments if you prefer)

N

---

<div class="post-metadata">

**Author:** ![Michael\_DeHaan](https://avatars.discourse-cdn.com/v4/letter/m/c4cdca/32.png) [@Michael\_DeHaan](https://forum.ansible.com/u/Michael_DeHaan)\
**Post date:** [February 26, 2013, 8:14pm UTC](https://forum.ansible.com/t/looping-over-group-vars-in-a-task-include-results-in-mis-interpolation-of-item/12424/13 "2013-02-26T20:14:54Z")

</div>

> In the ticket, dhozac says:
> 
> > The only case where it doesn't work is for include, which iterates the  
> > list and duplicates the tasks/plays for each item in the list

Daniel and I were discussing

tasks:  
&nbsp;&nbsp;&nbsp;&nbsp;- yum: name=$item state=present  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;with\_items: $packages

where $packages comes from group\_vars.

It's a little different from your include example.

It's also probably completely workable for what you want to do.

The "include with items" trick has to create task objects, and task  
objects need to be created the same for all the hosts in the group, so  
they don't have  
access to inventory variable information just yet.

Sorry, it's a little confusing, I agree... include+with\_items together  
is about as far on the 'advanced use case' side as it gets, thankfully  
🙂

---

<div class="post-metadata">

**Author:** ![seanosh](https://avatars.discourse-cdn.com/v4/letter/s/9fc29f/32.png) [@seanosh](https://forum.ansible.com/u/seanosh)\
**Post date:** [February 26, 2013, 9:41pm UTC](https://forum.ansible.com/t/looping-over-group-vars-in-a-task-include-results-in-mis-interpolation-of-item/12424/14 "2013-02-26T21:41:06Z")

</div>

Timely discussion, just running into this myself… am I correct in understanding that with an external inventory, something like this [http://ansible.cc/docs/playbooks2.html#advanced-task-includes](http://ansible.cc/docs/playbooks2.html#advanced-task-includes) isn’t possible? e.g.

# playbook.yml

---

<div class="post-metadata">

**Author:** ![Michael\_DeHaan](https://avatars.discourse-cdn.com/v4/letter/m/c4cdca/32.png) [@Michael\_DeHaan](https://forum.ansible.com/u/Michael_DeHaan)\
**Post date:** [February 26, 2013, 9:45pm UTC](https://forum.ansible.com/t/looping-over-group-vars-in-a-task-include-results-in-mis-interpolation-of-item/12424/15 "2013-02-26T21:45:53Z")

</div>

> Timely discussion, just running into this myself... am I correct in  
> understanding that with an external inventory, something like this  
> [http://ansible.cc/docs/playbooks2.html#advanced-task-includes](http://ansible.cc/docs/playbooks2.html#advanced-task-includes) isn't  
> possible? e.g.
> 
> # playbook.yml  
> ---  
> - include: wordpress.yml app=$item  
> &nbsp;&nbsp;with\_items: $apps
> 
> # wordpress.yml  
> ---  
> - name: check vars  
> &nbsp;&nbsp;template: src=checkvars.j2 dest=/tmp/ansible.vars
> 
> # checkvars.j2  
> {{ app }}
> 
> $apps is being set via a 'hostfile' script that consumes JSON data from an  
> API (very similar to ansible commander, e.g. not using hostvars or  
> group\_vars folders/files)... but when I examine the contents of  
> /tmp/ansible.vars on the remote machine, it has the entire list -  
> essentially appearing to just dump the contents of $apps into that file...

Correct, you'll have the same thing going on.

What you want to do instead is unroll that include, and reference the  
users or packages or other list inside the included file, like I  
referenced previously.

tasks:  
&nbsp;&nbsp;&nbsp;- action: whatever something=$item  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;with\_items: $apps

And that's fine.

Further, I'm not sure you were implying it, but you can't use Jinja2  
inside playbooks -- there's a good reason for this -- if you could,  
you couldn't parse the text, and they wouldn't be YAML anymore.

It used to technically work, but should not.

I'm kind of leaning on making "include + with\_items" undocumented as  
confusing as it actually is. We only actually documented it recently  
and it's being confusing to people with the limitations it has, and  
there are easy workarounds.

--Michael

---

<div class="post-metadata">

**Author:** ![seanosh](https://avatars.discourse-cdn.com/v4/letter/s/9fc29f/32.png) [@seanosh](https://forum.ansible.com/u/seanosh)\
**Post date:** [February 26, 2013, 9:54pm UTC](https://forum.ansible.com/t/looping-over-group-vars-in-a-task-include-results-in-mis-interpolation-of-item/12424/16 "2013-02-26T21:54:05Z")

</div>

When you say:

"Further, I’m not sure you were implying it, but you can’t use Jinja2  
inside playbooks – there’s a good reason for this – if you could,  
you couldn’t parse the text, and they wouldn’t be YAML anymore.

It used to technically work, but should not."

Are you saying that something like this wouldn’t or shouldn’t (or someday won’t) work:

---

<div class="post-metadata">

**Author:** ![Michael\_DeHaan](https://avatars.discourse-cdn.com/v4/letter/m/c4cdca/32.png) [@Michael\_DeHaan](https://forum.ansible.com/u/Michael_DeHaan)\
**Post date:** [February 26, 2013, 9:56pm UTC](https://forum.ansible.com/t/looping-over-group-vars-in-a-task-include-results-in-mis-interpolation-of-item/12424/17 "2013-02-26T21:56:13Z")

</div>

This works now.

---

<div class="post-metadata">

**Author:** ![seanosh](https://avatars.discourse-cdn.com/v4/letter/s/9fc29f/32.png) [@seanosh](https://forum.ansible.com/u/seanosh)\
**Post date:** [February 26, 2013, 10:01pm UTC](https://forum.ansible.com/t/looping-over-group-vars-in-a-task-include-results-in-mis-interpolation-of-item/12424/18 "2013-02-26T22:01:39Z")

</div>

Right. Perhaps my misunderstanding on what you meant by “use Jinja2 inside playbooks”… definitely aware and wouldn’t attempt to do something like that, because yes agreed, it should be YAML (and able to be parsed as such)…

Thanks for all your help…

– sean

---

<div class="post-metadata">

**Author:** ![Asad\_Khan](https://avatars.discourse-cdn.com/v4/letter/a/ac8455/32.png) [@Asad\_Khan](https://forum.ansible.com/u/Asad_Khan)\
**Post date:** [February 27, 2013, 7:24pm UTC](https://forum.ansible.com/t/looping-over-group-vars-in-a-task-include-results-in-mis-interpolation-of-item/12424/19 "2013-02-27T19:24:40Z")

</div>

Raises hand - what a coincidence, just experienced this myself and was about to post a message. In my case I was trying to loop over a created fact.