# Iterating over complex dictionary, but only when key exists

**URL:** <https://forum.ansible.com/t/iterating-over-complex-dictionary-but-only-when-key-exists/16570>\
**Category:** Ansible Project\
**Created:** [October 21, 2014, 1:17pm UTC](https://forum.ansible.com/t/iterating-over-complex-dictionary-but-only-when-key-exists/16570 "2014-10-21T13:17:26Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Dane](https://avatars.discourse-cdn.com/v4/letter/d/54ee81/32.png) [@Dane](https://forum.ansible.com/u/Dane)\
**Post date:** [October 21, 2014, 1:17pm UTC](https://forum.ansible.com/t/iterating-over-complex-dictionary-but-only-when-key-exists/16570/1 "2014-10-21T13:17:26Z")

</div>

Hello list,

I've got a complex dictionary I'm using to configure haproxy with,  
which looks something like this (non-relevant portions have been  
removed):

{  
&nbsp;&nbsp;&nbsp;&nbsp;"l7config": {  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"stats\_password": "AirwavesEyedrops",  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"frontends": [  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"frontend\_id": "6dcb7365-1b6c-4f24-a7ff-79590af2d536",  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"vip": "192.0.1.200",  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"service": "http",  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"port": 80,  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"backend\_id": "49cd3849-455c-4e5c-9e13-70cda7be0397"  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;},  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"frontend\_id": "01f99586-c14c-4c99-99e0-cf6effac3ee8",  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"vip": "192.0.1.200",  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"port": 443,  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"service": "http",  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"backend\_id": "49cd3849-455c-4e5c-9e13-70cda7be0397",  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"tls": {  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"pem": "-----BEGIN PRIVATE KEY-----\nMII  
...snip... 7w==\n-----END CERTIFICATE-----\n",  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"domain": "foobar.com"  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;],  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"backends": [  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;{  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;...  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;},  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;...  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;]  
&nbsp;&nbsp;&nbsp;&nbsp;}  
}

As you can see, I've got two frontend configurations, one which has a  
'tls' field and the other which doesn't. What I need to do is iterate  
over the frontends and discover which ones have the tls field, then  
write out the 'pem' certificates into a file.

I have this code in my playbook which isn't working as I think it  
should, as it always seems to just skip the task:

&nbsp;&nbsp;&nbsp;&nbsp;- name: Write out SSLs to files  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;copy: dest=/etc/ssl/{{item.tls.domain}}.pem owner=root  
group=root mode=0600 content={{item.tls.pem}}  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;with\_items:  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;- l7config.frontends  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;when: item.tls is defined

I suspect I'm on the right track, but I'm missing something blindingly  
obvious and stupid. I just need someone to point out where I'm being  
daft 🙂

Thanks

Dane

---

<div class="post-metadata">

**Author:** ![Dane](https://avatars.discourse-cdn.com/v4/letter/d/54ee81/32.png) [@Dane](https://forum.ansible.com/u/Dane)\
**Post date:** [October 22, 2014, 8:38am UTC](https://forum.ansible.com/t/iterating-over-complex-dictionary-but-only-when-key-exists/16570/2 "2014-10-22T08:38:08Z")

</div>

Ok, so after looking into this further I don't think with\_items can  
look 'inside' the dictionary, and I should instead be using  
with\_subelements. That doesn't appear to work either though:

- name: Write out SSLs to files  
&nbsp;&nbsp;copy: dest=/etc/ukfast/{{item.tls.domain}}.pem owner=root group=root  
mode=0600 content={{item.tls.pem}}  
&nbsp;&nbsp;with\_subelements:  
&nbsp;&nbsp;&nbsp;&nbsp;- l7config  
&nbsp;&nbsp;&nbsp;&nbsp;- frontends  
&nbsp;&nbsp;when: item.tls is defined

TASK: [layer7-lb | Write out SSLs to files] \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*  
fatal: [192.168.122.76] =\> subelements lookup expects a dictionary,  
got 'AirwavesEyedrops'

I'm not sure why it's pulling out the value of stats\_password instead  
of looking up the list 'frontends' inside the dictionary 'l7config'.

I've loaded my JSON into Python to confirm the types, too:

> > > type(a['l7config'])

\<type 'dict'\>

> > > type(a['l7config']['frontends'])

\<type 'list'\>

Regards

Dane

---

<div class="post-metadata">

**Author:** ![Michael\_Hoglan](https://avatars.discourse-cdn.com/v4/letter/m/a4c791/32.png) [@Michael\_Hoglan](https://forum.ansible.com/u/Michael_Hoglan)\
**Post date:** [October 22, 2014, 11:39pm UTC](https://forum.ansible.com/t/iterating-over-complex-dictionary-but-only-when-key-exists/16570/3 "2014-10-22T23:39:11Z")

</div>

Have you thought about breaking out some jinja expressions to manipulate the structure? In particular the selectrattr() filter.  
[http://jinja.pocoo.org/docs/dev/templates/](http://jinja.pocoo.org/docs/dev/templates/)

Here is a similar example I did

- hosts: localhost  
connection: local  
vars:  
config:  
frontend:
- id: 1  
value: test1  
tls:
- key: key1  
domain: domain1
- id: 2  
value: test2  
backend:
- id: 3
- id: 4  
tasks:
- debug: var=item  
with\_items: config.frontend
- debug: var=item  
with\_items: config.frontend | selectattr(“tls”, “defined”) | list

And then the output

(ansible)[ec2-user@ip-10-0-0-226 playbooks]$ ansible-playbook test\_withitems.yml

PLAY [localhost] \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*

GATHERING FACTS \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*  
ok: [localhost]

TASK: [debug var=item] \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*  
ok: [localhost] =\> (item={‘tls’: [{‘domain’: ‘domain1’, ‘key’: ‘key1’}], ‘id’: 1, ‘value’: ‘test1’}) =\> {  
“item”: {  
“id”: 1,  
“tls”: [  
{  
“domain”: “domain1”,  
“key”: “key1”  
}  
],  
“value”: “test1”  
}  
}  
ok: [localhost] =\> (item={‘id’: 2, ‘value’: ‘test2’}) =\> {  
“item”: {  
“id”: 2,  
“value”: “test2”  
}  
}

TASK: [debug var=item] \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*  
ok: [localhost] =\> (item={‘tls’: [{‘domain’: ‘domain1’, ‘key’: ‘key1’}], ‘id’: 1, ‘value’: ‘test1’}) =\> {  
“item”: {  
“id”: 1,  
“tls”: [  
{  
“domain”: “domain1”,  
“key”: “key1”  
}  
],  
“value”: “test1”  
}  
}

PLAY RECAP \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*  
localhost : ok=3 changed=0 unreachable=0 failed=0

That should give a list of all frontends that have a tls attribute defined.

Thanks!  
Michael

---

<div class="post-metadata">

**Author:** ![Dane](https://avatars.discourse-cdn.com/v4/letter/d/54ee81/32.png) [@Dane](https://forum.ansible.com/u/Dane)\
**Post date:** [October 23, 2014, 8:21am UTC](https://forum.ansible.com/t/iterating-over-complex-dictionary-but-only-when-key-exists/16570/4 "2014-10-23T08:21:33Z")

</div>

Hi Michael,

Thanks for the response. Your example does work for me, however  
looking at your example I've managed to figure out why my original  
example wasn't working:

&nbsp;&nbsp;&nbsp;&nbsp;- name: Write out SSLs to files  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;copy: dest=/etc/ssl/{{item.tls.domain}}.pem owner=root  
group=root mode=0600 content={{item.tls.pem}}  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;with\_items:  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;- l7config.frontends  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;when: item.tls is defined

Does not work, whereas:

&nbsp;&nbsp;&nbsp;&nbsp;- name: Write out SSLs to files  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;copy: dest=/etc/ssl/{{item.tls.domain}}.pem owner=root  
group=root mode=0600 content={{item.tls.pem}}  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;with\_items: l7config.frontends  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;when: item.tls is defined

Works fine. The with\_items can't be a list itself, as Ansible doesn't  
appear to 'see' inside the dictionary then, and just treats it as a  
string.

Didn't know about selectattr, so that's one to remember, but now I've  
got my original example working I'll stick with it as it seems more  
natural to me out of the two!

Thanks

Dane

---

<div class="post-metadata">

**Author:** ![Dane](https://avatars.discourse-cdn.com/v4/letter/d/54ee81/32.png) [@Dane](https://forum.ansible.com/u/Dane)\
**Post date:** [October 23, 2014, 8:28am UTC](https://forum.ansible.com/t/iterating-over-complex-dictionary-but-only-when-key-exists/16570/5 "2014-10-23T08:28:38Z")

</div>

Just for completeness in case anyone else stumbles across this issue,  
my final task was:

&nbsp;&nbsp;# Use long-form for task, Ansible issue #9172  
&nbsp;&nbsp;- name: Write out SSLs to files  
&nbsp;&nbsp;&nbsp;&nbsp;copy:  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;dest: /etc/ssl/{{item.tls.domain}}.pem  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;owner: root  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;group: root  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;mode: 0600  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;content: "{{item.tls.pem}}"  
&nbsp;&nbsp;&nbsp;&nbsp;with\_items: l7config.frontends  
&nbsp;&nbsp;&nbsp;&nbsp;when: item.tls is defined

Thanks

Dane

---

<div class="post-metadata">

**Author:** ![Michael\_Hoglan](https://avatars.discourse-cdn.com/v4/letter/m/a4c791/32.png) [@Michael\_Hoglan](https://forum.ansible.com/u/Michael_Hoglan)\
**Post date:** [October 23, 2014, 1:18pm UTC](https://forum.ansible.com/t/iterating-over-complex-dictionary-but-only-when-key-exists/16570/6 "2014-10-23T13:18:56Z")

</div>

When you do the list syntax for with\_items, it would have worked if you would have put in an expression.

- name: Write out SSLs to files  
copy: dest=/etc/ssl/{{item.tls.domain}}.pem owner=root  
group=root mode=0600 content={{item.tls.pem}}  
with\_items:
- “{{ l7config.frontends }}”  
when: item.tls is defined

Here is the output from the example I gave

- hosts: localhost  
connection: local  
vars:  
config:  
frontend:
- id: 1  
value: test1  
tls:
- key: key1  
domain: domain1
- id: 2  
value: test2  
backend:
- id: 3
- id: 4  
tasks:
- debug: var=item  
with\_items:
- “{{ config.frontend }}”  
when: item.tls is defined
- debug: var=item  
with\_items: config.frontend | selectattr(“tls”, “defined”) | list

And the result

(ansible)[ec2-user@ip-10-0-0-226 playbooks]$ ansible-playbook test\_withitems.yml

PLAY [localhost] \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*

GATHERING FACTS \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*  
ok: [localhost]

TASK: [debug var=item] \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*  
ok: [localhost] =\> (item={‘tls’: [{‘domain’: ‘domain1’, ‘key’: ‘key1’}], ‘id’: 1, ‘value’: ‘test1’}) =\> {  
“item”: {  
“id”: 1,  
“tls”: [  
{  
“domain”: “domain1”,  
“key”: “key1”  
}  
],  
“value”: “test1”  
}  
}  
skipping: [localhost] =\> (item={‘id’: 2, ‘value’: ‘test2’})

TASK: [debug var=item] \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*  
ok: [localhost] =\> (item={‘tls’: [{‘domain’: ‘domain1’, ‘key’: ‘key1’}], ‘id’: 1, ‘value’: ‘test1’}) =\> {  
“item”: {  
“id”: 1,  
“tls”: [  
{  
“domain”: “domain1”,  
“key”: “key1”  
}  
],  
“value”: “test1”  
}  
}

PLAY RECAP \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*  
localhost : ok=3 changed=0 unreachable=0 failed=0

I believe this occurs because in the single line syntax it assumes an expression already, which is why you can specify jinja filters without wrapping it all in a jinja {{ }} expression block. But when you do the long form / list form (don’t know what the correct terminology is), it doesn’t do expression blocks implicitly, you have to explicitly specify them.

Good news is, all these different ways of doing things is just different tools we can utilize depending on the situation, I find myself sometimes using jinja filters and other times using the standard with / when from ansible.

Thanks!  
Michael