In v1.2 with_items no longer iterates over a list of variables

Archives Ansible Project
1

In version 1.2, we cannot iterate over a list of variables. Ansible now returns the fatal error “with_items expects a list.” I need to reference a group variable to get a list of items. We’ve used this in a few different situations and now we need a another way to do this or a fix for this issue.

  • name: example of broken with_items issue
    hosts: all
    user: root
    vars:
    mail_aliases:
    root: sysadmin@example.com
    oracle: dbadmin@example.com
    tasks:
  • name: add mail aliases
    lineinfile: dest=/etc/aliases regexp=‘^${item}:’ line=“${item}:\t${mail_aliases.$item}”
    with_items: $mail_aliases
    when_set: $mail_aliases
    notify: newaliases
2

Cory:

In your example, it looks like mail_aliases is a hash. Should work if it’s a list, e.g.:

vars:
mail_aliases:

Also, I don’t think you need to dereference the variables anymore in “with_items”, so you can do:

with_items: mail_aliases

Lorin

3

Correct, with_items needs a list, not a hash.

4

This used to work with hashes too (iterating over hash keys). Is there
any reason for not supporting this anymore?

5

> Correct, with_items needs a list, not a hash.
>
This used to work with hashes too (iterating over hash keys). Is there
any reason for not supporting this anymore?

You can now use Python snippets here, try doing this to turn the dict into
a list with the values:

    with_items: mail_aliases.values()

6

What I’ve done a few times recently for k/v type things is use a list of “dicts” (in python speak):

- name: example of alternative with_items usage
  hosts: all
  user: root
  vars:
    mail_aliases:
      - {name: root, email: sysa...@example.com}
      - {name: oracle, email: dba...@example.com}
  tasks:
  - name: add mail aliases
    lineinfile:
      dest: /etc/aliases
      regexp: '^${item.name}:'
      line: "${item.name}:\t${$item.email}"
    with_items: $mail_aliases
    when_set: $mail_aliases
    notify: newaliases

(untested, but I think it should work)

This also has the flexibility of adding even more “values” while keeping the code very readable.

7

That’s funny as I never intended this, but .iteritems() works on both, so that follows.

I have nothing against checking if something is iterable instead of a list, but did not think anyone would have been doing it.

8

Yep, this will definitely work.

If you are in 1.2 you can avoid the ${ … } and just say {{ item.name }}

9

I was able to find a way around the problem by using the following reference:
with_items: $mail_aliases.keys()

10

I couldn’t find reference to either keys() or values() in the Jinja2 documentation

http://jinja.pocoo.org/docs/templates/

when I went hunting for something similar. Not being a Python hacker, and thinking variables in this context where pure Jinja2 variables, I ended up using the dictsort filter, so the original example could be done like so

lineinfile: dest=/etc/aliases regexp=‘^{{ item.0 }}:’ line=“{{ item.0 }}:\t{{ item.1 }}”
with_items: mail_aliases|dictsort

The nice thing about this approach is that you don’t lose the key => value mapping information in the original data.

K

11

It would be better to store the data as a list of hashes in that case.

12

True, for this play. But if you had an additional play where you wanted to
look up the alias for a particular user, you can't use the same data.
For one play, the data is most simply expressed as a list of hashes, but
for the other, its a hash. Duplicating the data is obviously problematic.

There may be some Python trick to pick out an element of a list by a key,
but its beyond me. I believe we can't use list comprehensions.

K

Kahlil (Kal) Hodgson GPG: C9A02289
Head of Technology (m) +61 (0) 4 2573 0382
DealMax Pty Ltd (w) +61 (0) 3 9008 5281

Suite 1415
401 Docklands Drive
Docklands VIC 3008 Australia

"All parts should go together without forcing. You must remember that
the parts you are reassembling were disassembled by you. Therefore,
if you can't get them together again, there must be a reason. By all
means, do not use a hammer." -- IBM maintenance manual, 1925

13

So what seems to occur to me here is that “with_items” is clearly a list iteration, but it might make sense to make a “with_dictionary” that allows you to get {{ item.key }} and {{ item.value }} seperately.

And then if the value was a dictionary, it would be like {{ item.value.somekey }}

Just a thought.

14

I like that. Much more obvious syntax :slight_smile:

15

Nice thought, although I would prefer using "with_keys", "{{ key.name
}}" and "{{ key.value }}". Same concept, better naming, imho. And it's
consistent with "with_items".

16

I had this problem a few months back, and wrote a pair of custom lookup plugins to do the job. (See below.) Was meaning to contribute it back but so far hadn't gotten around to it yet.

The examples below haven't been tested recently, so no guarantees they still work. But assuming they do, you can just put them in a directory, let's call it $library, define $ANSIBLE_LIBRARY to include it, then you use them via "with_keys" and "with_values".

Cheers and HTH

N

17

Also, I think that it would be more reasonable and closer to Ansible's
linear execution model to map YAML dictionaries to OrderedDict objects
(http://docs.python.org/2/library/collections.html#ordereddict-objects),
so that a 'with_keys' implementation iterates through the keys in the
order they are written in YAML code. The only problem is that
OrderedDict is available in Python version >= 2.7, but Ansible could
fallback to a backported version if available, or to an ordinary dict if
not.

What do you think?

18

I like with_keys; especially ordered! Little wary about falling back quietly to unordered. What about adding a companion attr? Say: unordered_ok=True ? Just looking to make the fallback explicit.

19

I’d suggest always sorting rather than having to request sorting.

20

“I have nothing against checking if something is iterable instead of a list, but did not think anyone would have been doing it.”

I had implemented this as part of a discussion with a colleague and then went back and found this discussion.

In part it was the “dict is iterable so it ought to be a suitable item iterator” reasoning that led to this.

It is not a good argument for it, but the change is only 3 lines of executable code…

Comments?
https://github.com/jkleckner/ansible/commit/9a18d60e6ef80bc6a25d29704c76fa93621d4830
https://github.com/jkleckner/ansible/tree/extend-with-items-allow-hash

Jim