Iterating over complex dictionary, but only when key exists

Archives Ansible Project
1

Hello list,

I've got a complex dictionary I'm using to configure haproxy with,
which looks something like this (non-relevant portions have been
removed):

{
    "l7config": {
        "stats_password": "AirwavesEyedrops",
        "frontends": [
            {
                "frontend_id": "6dcb7365-1b6c-4f24-a7ff-79590af2d536",
                "vip": "192.0.1.200",
                "service": "http",
                "port": 80,
                "backend_id": "49cd3849-455c-4e5c-9e13-70cda7be0397"
            },
            {
                "frontend_id": "01f99586-c14c-4c99-99e0-cf6effac3ee8",
                "vip": "192.0.1.200",
                "port": 443,
                "service": "http",
                "backend_id": "49cd3849-455c-4e5c-9e13-70cda7be0397",
                "tls": {
                    "pem": "-----BEGIN PRIVATE KEY-----\nMII
...snip... 7w==\n-----END CERTIFICATE-----\n",
                    "domain": "foobar.com"
                }
            }
        ],
        "backends": [
            {
                ...
            },
            ...
        ]
    }
}

As you can see, I've got two frontend configurations, one which has a
'tls' field and the other which doesn't. What I need to do is iterate
over the frontends and discover which ones have the tls field, then
write out the 'pem' certificates into a file.

I have this code in my playbook which isn't working as I think it
should, as it always seems to just skip the task:

    - name: Write out SSLs to files
      copy: dest=/etc/ssl/{{item.tls.domain}}.pem owner=root
group=root mode=0600 content={{item.tls.pem}}
      with_items:
        - l7config.frontends
      when: item.tls is defined

I suspect I'm on the right track, but I'm missing something blindingly
obvious and stupid. I just need someone to point out where I'm being
daft :slight_smile:

Thanks

Dane

2

Ok, so after looking into this further I don't think with_items can
look 'inside' the dictionary, and I should instead be using
with_subelements. That doesn't appear to work either though:

- name: Write out SSLs to files
  copy: dest=/etc/ukfast/{{item.tls.domain}}.pem owner=root group=root
mode=0600 content={{item.tls.pem}}
  with_subelements:
    - l7config
    - frontends
  when: item.tls is defined

TASK: [layer7-lb | Write out SSLs to files] ***********************************
fatal: [192.168.122.76] => subelements lookup expects a dictionary,
got 'AirwavesEyedrops'

I'm not sure why it's pulling out the value of stats_password instead
of looking up the list 'frontends' inside the dictionary 'l7config'.

I've loaded my JSON into Python to confirm the types, too:

type(a['l7config'])

<type 'dict'>

type(a['l7config']['frontends'])

<type 'list'>

Regards

Dane

3

Have you thought about breaking out some jinja expressions to manipulate the structure? In particular the selectrattr() filter.
http://jinja.pocoo.org/docs/dev/templates/

Here is a similar example I did

  • hosts: localhost
    connection: local
    vars:
    config:
    frontend:
  • id: 1
    value: test1
    tls:
  • key: key1
    domain: domain1
  • id: 2
    value: test2
    backend:
  • id: 3
  • id: 4
    tasks:
  • debug: var=item
    with_items: config.frontend
  • debug: var=item
    with_items: config.frontend | selectattr(“tls”, “defined”) | list

And then the output

(ansible)[ec2-user@ip-10-0-0-226 playbooks]$ ansible-playbook test_withitems.yml

PLAY [localhost] **************************************************************

GATHERING FACTS ***************************************************************
ok: [localhost]

TASK: [debug var=item] ********************************************************
ok: [localhost] => (item={‘tls’: [{‘domain’: ‘domain1’, ‘key’: ‘key1’}], ‘id’: 1, ‘value’: ‘test1’}) => {
“item”: {
“id”: 1,
“tls”: [
{
“domain”: “domain1”,
“key”: “key1”
}
],
“value”: “test1”
}
}
ok: [localhost] => (item={‘id’: 2, ‘value’: ‘test2’}) => {
“item”: {
“id”: 2,
“value”: “test2”
}
}

TASK: [debug var=item] ********************************************************
ok: [localhost] => (item={‘tls’: [{‘domain’: ‘domain1’, ‘key’: ‘key1’}], ‘id’: 1, ‘value’: ‘test1’}) => {
“item”: {
“id”: 1,
“tls”: [
{
“domain”: “domain1”,
“key”: “key1”
}
],
“value”: “test1”
}
}

PLAY RECAP ********************************************************************
localhost : ok=3 changed=0 unreachable=0 failed=0

That should give a list of all frontends that have a tls attribute defined.

Thanks!
Michael

4

Hi Michael,

Thanks for the response. Your example does work for me, however
looking at your example I've managed to figure out why my original
example wasn't working:

    - name: Write out SSLs to files
      copy: dest=/etc/ssl/{{item.tls.domain}}.pem owner=root
group=root mode=0600 content={{item.tls.pem}}
      with_items:
        - l7config.frontends
      when: item.tls is defined

Does not work, whereas:

    - name: Write out SSLs to files
      copy: dest=/etc/ssl/{{item.tls.domain}}.pem owner=root
group=root mode=0600 content={{item.tls.pem}}
      with_items: l7config.frontends
      when: item.tls is defined

Works fine. The with_items can't be a list itself, as Ansible doesn't
appear to 'see' inside the dictionary then, and just treats it as a
string.

Didn't know about selectattr, so that's one to remember, but now I've
got my original example working I'll stick with it as it seems more
natural to me out of the two!

Thanks

Dane

5

Just for completeness in case anyone else stumbles across this issue,
my final task was:

  # Use long-form for task, Ansible issue #9172
  - name: Write out SSLs to files
    copy:
      dest: /etc/ssl/{{item.tls.domain}}.pem
      owner: root
      group: root
      mode: 0600
      content: "{{item.tls.pem}}"
    with_items: l7config.frontends
    when: item.tls is defined

Thanks

Dane

6

When you do the list syntax for with_items, it would have worked if you would have put in an expression.

  • name: Write out SSLs to files
    copy: dest=/etc/ssl/{{item.tls.domain}}.pem owner=root
    group=root mode=0600 content={{item.tls.pem}}
    with_items:
  • “{{ l7config.frontends }}”
    when: item.tls is defined

Here is the output from the example I gave

  • hosts: localhost
    connection: local
    vars:
    config:
    frontend:
  • id: 1
    value: test1
    tls:
  • key: key1
    domain: domain1
  • id: 2
    value: test2
    backend:
  • id: 3
  • id: 4
    tasks:
  • debug: var=item
    with_items:
  • “{{ config.frontend }}”
    when: item.tls is defined
  • debug: var=item
    with_items: config.frontend | selectattr(“tls”, “defined”) | list

And the result

(ansible)[ec2-user@ip-10-0-0-226 playbooks]$ ansible-playbook test_withitems.yml

PLAY [localhost] **************************************************************

GATHERING FACTS ***************************************************************
ok: [localhost]

TASK: [debug var=item] ********************************************************
ok: [localhost] => (item={‘tls’: [{‘domain’: ‘domain1’, ‘key’: ‘key1’}], ‘id’: 1, ‘value’: ‘test1’}) => {
“item”: {
“id”: 1,
“tls”: [
{
“domain”: “domain1”,
“key”: “key1”
}
],
“value”: “test1”
}
}
skipping: [localhost] => (item={‘id’: 2, ‘value’: ‘test2’})

TASK: [debug var=item] ********************************************************
ok: [localhost] => (item={‘tls’: [{‘domain’: ‘domain1’, ‘key’: ‘key1’}], ‘id’: 1, ‘value’: ‘test1’}) => {
“item”: {
“id”: 1,
“tls”: [
{
“domain”: “domain1”,
“key”: “key1”
}
],
“value”: “test1”
}
}

PLAY RECAP ********************************************************************
localhost : ok=3 changed=0 unreachable=0 failed=0

I believe this occurs because in the single line syntax it assumes an expression already, which is why you can specify jinja filters without wrapping it all in a jinja {{ }} expression block. But when you do the long form / list form (don’t know what the correct terminology is), it doesn’t do expression blocks implicitly, you have to explicitly specify them.

Good news is, all these different ways of doing things is just different tools we can utilize depending on the situation, I find myself sometimes using jinja filters and other times using the standard with / when from ansible.

Thanks!
Michael