# Writing/Augmenting Host Variables from a Playbook. **URL:** \ **Category:** Ansible Project\ **Created:** [April 27, 2020, 8:12am UTC](https://forum.ansible.com/t/writing-augmenting-host-variables-from-a-playbook/32788 "2020-04-27T08:12:01Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![Marc\_Haber](https://avatars.discourse-cdn.com/v4/letter/m/a3d4f5/32.png) [@Marc\_Haber](https://forum.ansible.com/u/Marc_Haber)\ **Post date:** [April 27, 2020, 8:12am UTC](https://forum.ansible.com/t/writing-augmenting-host-variables-from-a-playbook/32788/1 "2020-04-27T08:12:01Z")
Let's assume I have a service that can listen on different combinations or host/port: ---
**Author:** ![racke](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/racke/32/4767_2.png) [@racke](https://forum.ansible.com/u/racke)\ **Post date:** [April 28, 2020, 7:48am UTC](https://forum.ansible.com/t/writing-augmenting-host-variables-from-a-playbook/32788/2 "2020-04-28T07:48:23Z")
Hello Marc, nice to see you here :-). This is certainly doable: ---
**Author:** ![Marc\_Haber](https://avatars.discourse-cdn.com/v4/letter/m/a3d4f5/32.png) [@Marc\_Haber](https://forum.ansible.com/u/Marc_Haber)\ **Post date:** [May 2, 2020, 1:35pm UTC](https://forum.ansible.com/t/writing-augmenting-host-variables-from-a-playbook/32788/3 "2020-05-02T13:35:18Z")
Hi Stefan, thanks for your answer. I busied myself with working around the issue temporarily because I had to get a project forward. > \> Let's assume I have a service that can listen on different combinations > \> or host/port: > \> > > Hello Marc, > > nice to see you here :-). This is certainly doable: > > --- > - hosts: all > >   vars: >     host: >       service: >         hostname: "google.com" >         ports: >           - "80" >           - "443" >   tasks: >     - name: Get DNS records >       set\_fact: >         ip\_addresses: "{{ ip\_addresses | default() + lookup('dig', host.service.hostname + '/' + item, wantlist=True) }}" >       loop: >         - A >         - AAAA >        when: "'listen' not in host.service" This sets a local fact to the list of IP addreses pulled from DNS. Adding the IPv6 addresses is an easy enogh exercise. >     - name: Determine all combinations of IP address and ports >       set\_fact: >         ips\_ports: "{{ ip\_addresses | product(host.service.ports) | list }}" >       when: "'listen' not in host.service" That's magic. Cute, indeed. This returns a list, alternating between IP adress and port, like [ip1, port1, ip2, port2, ip3, port3] >     - name: Turn list members into dictionaries >       set\_fact: >         host: >           service: >             listen: "{{ host.service.listen | default() + [{ 'ip': item[0], 'port': item[1] }] }}" >       loop: "{{ ips\_ports }}" >       when: ips\_ports is defined This is where you're losing me. You're iterating over an array and get array members, so item will be "ip1" in the first iteration, "port1" in the second, "ip2" in the third. How can you index into an array element??? Anyway, things have become a bit more complicated since my original request. This is what I have now: service:   sites:     default:       protocols:       - "http"       - "https"   listen:   - ip: "2a01:4f8:161:3541::32:100"     port: "80"     protocol: "http"   - ip: "2a01:4f8:161:3541::32:100"     port: "443"     protocol: "https" The templates pull site configuration from the "sites" part of the definition; the listen configuration from the "listen" part of the definition. This is a strongly simplified example with reality being much more complex, I just reduced this to the relevant parts. This is, however, from a working setup (and yes, it's IPv6 only). For better readability, I'd like to write this like: service:   sites:     default:       protocols:       - "http"       - "https"       listen:       - ip: "2a01:4f8:161:3541::32:100" With the possibility of defining explicit ports and protocols in the listen clause. I think to massage one format into the other one in a task is awfully hard, and to use local facts makes it impossible to override the automatism by explicitly writing the desired result to inventory proper. Can a custom inventory plugin access what the previously running inventory plugins have parsed, and can it augment structure that was build by the predecessors? This way, I could have all processing power and flexibility of imperative programming. I could think of a gazillion of other places where this could be useful to simplify my templates \_AND\_ my inventory. Having this done inside ansible would allow me to take advantage of the inventory reading logic that is already present in ansible. I could write a preprocessor writing out the "augmented inventory" before ansible is started, but I'd have to manually process the inventory file -and- the contents of the host\_vars and group\_vars directories. I'd like to avoid this. Greetings Marc ---
**Author:** ![racke](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/racke/32/4767_2.png) [@racke](https://forum.ansible.com/u/racke)\ **Post date:** [May 2, 2020, 2:12pm UTC](https://forum.ansible.com/t/writing-augmenting-host-variables-from-a-playbook/32788/4 "2020-05-02T14:12:11Z")
> Hi Stefan, > > thanks for your answer. I busied myself with working around the issue > temporarily because I had to get a project forward. > > > > Let's assume I have a service that can listen on different combinations > > > or host/port: > > > > Hello Marc, > > > > nice to see you here :-). This is certainly doable: > > > > --- > > - hosts: all > > > >   vars: > >     host: > >       service: > >         hostname: "google.com" > >         ports: > >           - "80" > >           - "443" > >   tasks: > >     - name: Get DNS records > >       set\_fact: > >         ip\_addresses: "{{ ip\_addresses | default() + lookup('dig', host.service.hostname + '/' + item, wantlist=True) }}" > >       loop: > >         - A > >         - AAAA > >        when: "'listen' not in host.service" > > This sets a local fact to the list of IP addreses pulled from DNS. > Adding the IPv6 addresses is an easy enogh exercise. It already contains the IPv6 addresses (AAAA record). > >     - name: Determine all combinations of IP address and ports > >       set\_fact: > >         ips\_ports: "{{ ip\_addresses | product(host.service.ports) | list }}" > >       when: "'listen' not in host.service" > > That's magic. Cute, indeed. This returns a list, alternating between IP > adress and port, like [ip1, port1, ip2, port2, ip3, port3] It contains all IP and port combinations as nested list: [[ip1,port1],[ip1,port2],[ip2,port1]] > >     - name: Turn list members into dictionaries > >       set\_fact: > >         host: > >           service: > >             listen: "{{ host.service.listen | default() + [{ 'ip': item[0], 'port': item[1] }] }}" > >       loop: "{{ ips\_ports }}" > >       when: ips\_ports is defined > > This is where you're losing me. You're iterating over an array and get > array members, so item will be "ip1" in the first iteration, "port1" in > the second, "ip2" in the third. How can you index into an array > element??? See above, each entry is a list of IP (item[0]) and port (item[1]). Regards          Racke ---
**Author:** ![Marc\_Haber](https://avatars.discourse-cdn.com/v4/letter/m/a3d4f5/32.png) [@Marc\_Haber](https://forum.ansible.com/u/Marc_Haber)\ **Post date:** [May 9, 2020, 5:36am UTC](https://forum.ansible.com/t/writing-augmenting-host-variables-from-a-playbook/32788/5 "2020-05-09T05:36:05Z")
> \>\>\> Let's assume I have a service that can listen on different combinations > \>\>\> or host/port: > \>\>\> > \>\> > \>\> Hello Marc, > \>\> > \>\> nice to see you here :-). This is certainly doable: > \>\> > \>\> --- > \>\> - hosts: all > \>\> > \>\> vars: > \>\> host: > \>\> service: > \>\> hostname: "google.com" > \>\> ports: > \>\> - "80" > \>\> - "443" > \>\> tasks: > \>\> - name: Get DNS records > \>\> set\_fact: > \>\> ip\_addresses: "{{ ip\_addresses | default() + lookup('dig', host.service.hostname + '/' + item, wantlist=True) }}" > \>\> loop: > \>\> - A > \>\> - AAAA > \>\> when: "'listen' not in host.service" > \> > \> This sets a local fact to the list of IP addreses pulled from DNS. > \> Adding the IPv6 addresses is an easy enogh exercise. > > It already contains the IPv6 addresses (AAAA record). Idiot me, of course, I missed the loop. > \>\> - name: Determine all combinations of IP address and ports > \>\> set\_fact: > \>\> ips\_ports: "{{ ip\_addresses | product(host.service.ports) | list }}" > \>\> when: "'listen' not in host.service" > \> > \> That's magic. Cute, indeed. This returns a list, alternating between IP > \> adress and port, like [ip1, port1, ip2, port2, ip3, port3] > > It contains all IP and port combinations as nested list: > > [[ip1,port1],[ip1,port2],[ip2,port1]] Now it all makes sense to me. Thanks for explaining. I'm still wondering whether it would be possible for a more complex of variable data definition. > \> Can a custom inventory plugin access what the previously running > \> inventory plugins have parsed, and can it augment structure that was > \> build by the predecessors? This way, I could have all processing power > \> and flexibility of imperative programming. I could think of a gazillion > \> of other places where this could be useful to simplify my templates > \> \_AND\_ my inventory. > \> > \> Having this done inside ansible would allow me to take advantage of the > \> inventory reading logic that is already present in ansible. I could > \> write a preprocessor writing out the "augmented inventory" before > \> ansible is started, but I'd have to manually process the inventory file > \> -and- the contents of the host\_vars and group\_vars directories. I'd like > \> to avoid this. Any idea whether this would work? Greetings Marc ---
**Author:** ![racke](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/racke/32/4767_2.png) [@racke](https://forum.ansible.com/u/racke)\ **Post date:** [May 9, 2020, 5:49am UTC](https://forum.ansible.com/t/writing-augmenting-host-variables-from-a-playbook/32788/6 "2020-05-09T05:49:46Z")
> > > > > Let's assume I have a service that can listen on different combinations > > > > > or host/port: > > > > > > > > Hello Marc, > > > > > > > > nice to see you here :-). This is certainly doable: > > > > > > > > --- > > > > - hosts: all > > > > > > > >   vars: > > > >     host: > > > >       service: > > > >         hostname: "google.com" > > > >         ports: > > > >           - "80" > > > >           - "443" > > > >   tasks: > > > >     - name: Get DNS records > > > >       set\_fact: > > > >         ip\_addresses: "{{ ip\_addresses | default() + lookup('dig', host.service.hostname + '/' + item, wantlist=True) }}" > > > >       loop: > > > >         - A > > > >         - AAAA > > > >        when: "'listen' not in host.service" > > > > > > This sets a local fact to the list of IP addreses pulled from DNS. > > > Adding the IPv6 addresses is an easy enogh exercise. > > > > It already contains the IPv6 addresses (AAAA record). > > Idiot me, of course, I missed the loop. > > > > >     - name: Determine all combinations of IP address and ports > > > >       set\_fact: > > > >         ips\_ports: "{{ ip\_addresses | product(host.service.ports) | list }}" > > > >       when: "'listen' not in host.service" > > > > > > That's magic. Cute, indeed. This returns a list, alternating between IP > > > adress and port, like [ip1, port1, ip2, port2, ip3, port3] > > > > It contains all IP and port combinations as nested list: > > > > [[ip1,port1],[ip1,port2],[ip2,port1]] > > Now it all makes sense to me. Thanks for explaining. I'm still wondering > whether it would be possible for a more complex of variable data > definition. It is possible, but the Jinja expressions may become really complex which you might want to prevent. > > > Can a custom inventory plugin access what the previously running > > > inventory plugins have parsed, and can it augment structure that was > > > build by the predecessors? This way, I could have all processing power > > > and flexibility of imperative programming. I could think of a gazillion > > > of other places where this could be useful to simplify my templates > > > \_AND\_ my inventory. > > > > > > Having this done inside ansible would allow me to take advantage of the > > > inventory reading logic that is already present in ansible. I could > > > write a preprocessor writing out the "augmented inventory" before > > > ansible is started, but I'd have to manually process the inventory file > > > -and- the contents of the host\_vars and group\_vars directories. I'd like > > > to avoid this. > > Any idea whether this would work? You could also write a custom module in Python which can transform the structure as you wish. You can call your custom module in the first task to achieve the augmenting. Regards          Racke ---
**Author:** ![Marc\_Haber](https://avatars.discourse-cdn.com/v4/letter/m/a3d4f5/32.png) [@Marc\_Haber](https://forum.ansible.com/u/Marc_Haber)\ **Post date:** [May 9, 2020, 6:47am UTC](https://forum.ansible.com/t/writing-augmenting-host-variables-from-a-playbook/32788/7 "2020-05-09T06:47:11Z")
\_That\_ sounds totally interesting, can you point me to some example code please? Greetings Marc ---
**Author:** ![racke](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/racke/32/4767_2.png) [@racke](https://forum.ansible.com/u/racke)\ **Post date:** [May 9, 2020, 6:53am UTC](https://forum.ansible.com/t/writing-augmenting-host-variables-from-a-playbook/32788/8 "2020-05-09T06:53:37Z")
I don't have a custom module around which I could share here. Searching the web for "ansible custom module" should give you plenty of insights though. Regards            Racke ---
**Author:** ![Marc\_Haber](https://avatars.discourse-cdn.com/v4/letter/m/a3d4f5/32.png) [@Marc\_Haber](https://forum.ansible.com/u/Marc_Haber)\ **Post date:** [May 9, 2020, 7:09am UTC](https://forum.ansible.com/t/writing-augmenting-host-variables-from-a-playbook/32788/9 "2020-05-09T07:09:58Z")
Are you trying to say that a plain custom module called from a task (a) has access to the full inventory including all variables and (b) can write to it with following tasks seeing the changes it did? Greetings Marc ---
**Author:** ![racke](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/racke/32/4767_2.png) [@racke](https://forum.ansible.com/u/racke)\ **Post date:** [May 9, 2020, 7:21am UTC](https://forum.ansible.com/t/writing-augmenting-host-variables-from-a-playbook/32788/10 "2020-05-09T07:21:34Z")
That's a good question :-). But you can also write filter plugins which is probably a better idea. This would allow you do augment your data structure with "{{ data | augment }}" You can find an example here: [https://blog.oddbit.com/post/2019-04-25-writing-ansible-filter-plugins/](https://blog.oddbit.com/post/2019-04-25-writing-ansible-filter-plugins/) Regards          Racke ---
**Author:** ![Marc\_Haber](https://avatars.discourse-cdn.com/v4/letter/m/a3d4f5/32.png) [@Marc\_Haber](https://forum.ansible.com/u/Marc_Haber)\ **Post date:** [May 9, 2020, 8:28am UTC](https://forum.ansible.com/t/writing-augmenting-host-variables-from-a-playbook/32788/11 "2020-05-09T08:28:49Z")
I am not sure whether this is a better idea, making the augmentation dependent to the actual task being in quesiton. An independent approach would, for example, allow people to use generic apache, letsencrypt, proxy and DNS modules from The Galaxy while just writing the URL and other web site data in the host definition, with independent code taking the job of converting the simple host definition into input data structures the generic modules can grok. Greetings Marc