Omit and the dict lookup

Get Help
1

I’m seeing unexpected effects when using the omit value with the ansible.builtin.dict lookup. I think I understand what it’s doing, but I’m coming up short in the why department.

Here’s a sample playbook that demonstrates what happens when NOT using the omit magic value.

---
- name: Ansible omit games
  hosts: localhost
  gather_facts: false
  vars:
    ords:
      ones:   11111
      twos:   22222
      threes: 33333
      fours:  44444
      fives:  55555
    evens:
      - twos
      - fours
  tasks:
    - name: Print data
      ansible.builtin.debug:
        msg:
          - ords:  '{{ ords  }}'
          - evens: '{{ evens }}'
          - evens_x_omit:        '{{ evens | product(["omit"]) }}'
          - same_as_a_dict: '{{ dict(evens | product(["omit"])) }}'
          - ords_combined_with_that_dict:
             '{{ ords | combine(dict(evens | product(["omit"]))) }}'

Note in particular that "omit" above is just a 4-leter double-quoted string in every case, not the special omit value. The above produces these totally expected results:

  msg:
  - ords:
      fives: 55555
      fours: 44444
      ones: 11111
      threes: 33333
      twos: 22222
  - evens:
    - twos
    - fours
  - evens_x_omit:
    - - twos
      - omit
    - - fours
      - omit
  - same_as_a_dict:
      fours: omit
      twos: omit
  - ords_combined_with_that_dict:
      fives: 55555
      fours: omit
      ones: 11111
      threes: 33333
      twos: omit

So far so good. Now remove the double-quotes from all the "omit"s in that last task so we get the special omit value instead. And remove uninteresting ords and evens to same space.

    - name: Print data
      ansible.builtin.debug:
        msg:
          - evens_x_omit:        '{{ evens | product([omit]) }}'
          - same_as_a_dict: '{{ dict(evens | product([omit])) }}'
          - ords_combined_with_that_dict:
             '{{ ords | combine(dict(evens | product([omit]))) }}'

and that produces this output:

  msg:
  - evens_x_omit:
    - - twos
      - __omit_place_holder__baa56393a508bede3535cf10e0a2430226693e95
    - - fours
      - __omit_place_holder__baa56393a508bede3535cf10e0a2430226693e95
  - same_as_a_dict: {}
  - ords_combined_with_that_dict:
      fives: 55555
      ones: 11111
      threes: 33333

In this case, evens_x_omit contains exactly what one would expect: each value from evens paired with the special omit value.

However, feeding that to the [ansible.builtin.]dict lookup to produce same_as_a_dict has the surprising effect of omitting the key:values pairs where the value is the special omit value! I had thought that omit only had an effect when used in module parameters. But here we are with an apparently empty dict. I didn’t see that coming.

If that’s not surprising enough, if we use that “empty” dict in the combine filter, its emptiness somehow manifests in such a way to cause the combined dict’s twos and fours be omitted! It’s as if we’ve tricked the combine filter to have the effect that ansible.utils.remove_keys was designed for.

Maybe there’s something about omitting key:value pairs from dictionaries that takes place when closing a “{{}}” context?
Something that doesn’t apply to list values?
Is this a recent thing, or has it been this way for a long time?
Is this intentional? I don’t recall every seeing this behavior documented before.

Interested to know what’s really going on here. Thanks for watchin’.

2

Before 2.19, when omit is the value of a key, the key is removed. This only happens with dicts, no other data structures.

As of 2.19, omit, now not a special text placeholder, gets removed from all data structures, never showing up with the __omit* placeholder text.

As for:

I mean yes, you are feeding that to debug, and the removal in pre 2.19 is recursive. msg is a “module” parameter, and thus, the recursive removal will nuke the keys associated with the omit values.

3

Oh! That’s a couple more things I hadn’t considered: recursion/data structure depth and Ansible version. I’ve come to ignore that for the entertainment value; it delivers!

Thanks for the heads-up.

4

I see different results in 2.20.1

- debug:
    msg: "{{ evens | product([omit]) }}"
  vars:
    evens: [twos, fours]

ok: [localhost] =>
    msg:
    -   - twos
    -   - fours

- debug:
    msg: "{{ d1 | combine(d2) }}"
  vars:
    d1: {x: foo, y: bar, z: baz}
    d2: {x: foo, y: omit}

ok: [localhost] =>
    msg:
        x: foo
        y: omit
        z: baz

[source code]