with_items passed individually and as list

Archives Ansible Project
1

Hello,

I have a simple playbook that just defines an APT task to purge
packages provided as with_items list.

  tasks:
    - name: remove packages
      apt: pkg={{item}} state=absent purge=yes update_cache=no
      with_items:
        - puppet
        - puppet-common
        […]

Running this across a set of hosts, I got the following,
inconsistent output (abridged):

  ok: [diamond.madduck.net] => (item=puppet)
  ok: [arnold.madduck.net] => (item=puppet)
  ok: [arnold.madduck.net] => (item=puppet-common)
  ok: [diamond.madduck.net] => (item=puppet-common)
  […]
  changed: [jugband.madduck.net] => (item=puppet,puppet-common,facter,libopenssl-ruby,libshadow-ruby1.8,libxmlrpc-ruby,libruby,libruby1.8,libaugeas-ruby1.8,libdpkg-ruby1.8,libhttp-access2-ruby1.8,libuconv-ruby1.8,libxml-parser-ruby1.8,ri1.8,facter,ruby-json,ruby1.8,ruby-shadow)
  ok: [vizier.madduck.net] => (item=puppet,puppet-common,facter,libopenssl-ruby,libshadow-ruby1.8,libxmlrpc-ruby,libruby,libruby1.8,libaugeas-ruby1.8,libdpkg-ruby1.8,libhttp-access2-ruby1.8,libuconv-ruby1.8,libxml-parser-ruby1.8,ri1.8,facter,ruby-json,ruby1.8,ruby-shadow)

That is to say: some hosts iterated through the package list
one-by-one, while others received a list.

What determines this difference? How can I always pass lists?

Thanks,

2

I suspect you are running recent 1.3 code against a 1.2 module directory and the module doesn’t have the WITH_ITEMS_WANTS_LIST string in it, which is the new way we tell modules what they can do in series.

3

also sprach Michael DeHaan <michael@ansibleworks.com> [2013.06.19.1713 +0200]:

I suspect you are running recent 1.3 code against a 1.2 module
directory and the module doesn't have the WITH_ITEMS_WANTS_LIST
string in it, which is the new way we tell modules what they can
do in series.

I am running only from source and there are no modules installed
anywhere else.

ansible-playbook 1.3 (devel 0bd92d8196) last updated 2013/06/19 17:39:33 (GMT +200)

I verified that the module that gets executed also contains
WITH_ITEMS_WANTS_LIST

On a playbook run, some hosts produce one ok: line for each
with_list item, and other hosts produce one ok: for all items,
specifying a list.

There are no differences I can see between the hosts, i.e. they all
run Debian 7.0 with identical Python versions and package sets.

Very curiously, the set of hosts for which the APT runs will display
individually and the set of hosts displaying lists is non-constant,
i.e. it differs between runs.

It's weird, isn't it? Could this be a bug or maybe a possible
control path in the code wasn't updated?

I investigated a bit and especially had a look at what happens at
lib/ansible/runner/__init__.py:419, and I think I found a symptom:

self.module_with_list starts off False for the first batch of $forks
hosts, and then turns True, so hosts 6 through n will get lists
passed to the APT module, but hosts 1 through 5 won't.

I suspect that this is a multiprocessing problem. How does the
runner piece this all together? How can I help debug this further?

Cheers,

4

​possibly https://github.com/ansible/ansible/pull/3267 ?​

5

Nah, that’s just a documentation gotcha.

I also am not seeing the expected behavior of a single module execution over a list of items. Digging through the code, I saw that in runner/init.py module_with_list was set to True after the module was copied over. I got in over my head and couldn’t figure out how it was supposedly / or should be set prior to executing the copy step. Martin seems to have made more progress than me! :slight_smile:

6

I suspect you too are pointing at 1.2 module paths, but I’ll test it.

The 1.3 modules have a WITH_ITEMS_AS_LIST in their resource that allows them to know to merge things together and do everything in one pass.

7

I’ve confirmed the 1.3 pull request that intended to streamline this doesn’t quite work.

Working on this, stay tuned :slight_smile:

8

Commit reverted, back to normal.

9

mybad! modules worked but I didn’t check that they worked with a list. I didn’t realize they would fall back to single item.