Aggregating data from complex hierarchal sources

sirklezero
Fri, 04 Apr 2025 19:32:54 +0000
Get Help

I’m developing an Ansible playbook to help me automate the management of several dozen zones in Bind using the Community nsupdate module https://docs.ansible.com/ansible/latest/collections/community/general/nsupdate_module.html. Generally speaking, things have gone really well but I’ve found myself in a pickle with the data structure I’ve designed to store the configuration of the zone. I am pretty new to Ansible, and have a software engineering background, but I feel like either my data structure needs a tweak or my understanding of how to handle looping and combining data just isn’t there (probably both).

First, here’s the relevant portion of my inventory; nothing too surprising here:

nameservers:
  children:
    ns_masters:
    ns_slaves:

ns_masters:
  hosts:
    ns1.mydomain.com:
      hostname: ns1
      ip_address: x.x.x.x
      interface_name: eno0
      type: master

ns_slaves:
  hosts:
    ns2.mydomain.com:
      hostname: ns2
      ip_address: x.x.x.x
      interface_name: eno0
      type: slave

Next, I have created a template for zone files. My goal was to have a format that’s easy to read and maintain, and that intuitively follows the standard for zone files. Each file is named for the zone it manages.

As an example:

---
mydomain_com:
  - zone: mydomain.com
    default_ttl: 5m
    soa:
      primary_nameserver: ns1
      admin: me.mydomain.com
      serial: 2025033100
      refresh: 30m
      retry: 15m
      expire: 4w
      minimum: 2m
    ns_records:
      - ns1
      - ns2
    a_records:
      - {label: "@", rdata: "x.x.x.x"}
      - {label: "www", rdata: "x.x.x.x"}
      - {label: "ns1", rdata: "x.x.x.x"}
      - {label: "ns2", rdata: "x.x.x.x"}
    cname_records:
      - {label: "my-cname", rdata: "www.mydomain.com"}
      - {label: "another-cname", rdata: "www.mydomain.com"}

Next, I have a configuration file that holds information about the overall Bind configuration, including a property that aggregates the separate zones into a massive list (at least I’m pretty sure it’s a list and not a dictionary). Or is it a list of dictionaries in this case?

# Bind9 user and group that named runs under
bind9_user: bind
bind9_group: bind

# Bind9 configuration paths
bind_directory: "/etc/bind"
keys_directory: "/etc/bind/keys"
zones_directory: "/etc/bind/zones"
cache_directory: "/var/cache/bind"

# Bind9 acls
acl:
  - name: ACL1
    networks:
      - localhost
      - localnets
  - name: ACL2
    networks:
      - localhost
      - localnets

zones: >
  {{ mydomain_com
  | union (mydomain2_com) }}
  | union (mydomain3_com) }}
  | union (mydomain4_com) }}
  | union (mydomain5_com) }}
  | union (mydomain6_com) }}
  | union (mydomain7_com) }}
  | union (mydomain8_com) }}
  | union (mydomain9_com) }}
  | union (mydomain10_com) }}

This configuration file is loaded into the playbook, and the “zones” property is populated by all of the zone specific configurations defined in the individual zone files. Up to this point, everything seems to be working perfectly fine, and as you’d expect.

However, I am having an issue iterating over the zones and their associated records (a, cname, etc.). The playbook is really pretty simple and I don’t have any issues with it except for the task that needs to iterate the records and issue the “nsupdate” module executions.

The task as I’ve landed on it looks like this:

tasks:
  - name: Updating authoritative forward zone files
    ansible.builtin.debug:
      msg:
        - "{{ item }}"
    loop: "{{ zones | map(attribute='a_records') | flatten(levels=1) }} "
    loop_control:
      label: "{{ item.label }}"
    when:
      - inventory_hostname in groups['ns_masters']
    tags:
      - zone_update

This task only executes against my primary name servers, and only when I’m performing an actual zone update (via tag and when); it works great, and outputs the following debug information:

ok: [name-of-my-dns-server-redacted] => (item=www) => {
    "msg": [
        {
            "label": "www",
            "rdata": "x.x.x.x"
        }
    ]
}

Unfortunately, I need more than just the “label” and “rdata” to run the nsupdate module. Ultimately I need the output of this task to look like this:

ok: [name-of-my-dns-server-redacted] => (item=www) => {
    "msg": [
        {
            "label": "www",
            "rdata": "x.x.x.x"
            "zone": "mydomain.com"
            "server": "my-nameserver-ip-address-goes-here"
        }
    ]
}

The zone value should come from the zone property of the zone configuration file, and the server should come from the inventory. In theory, each of these is trivial to get to, but I’m really really stuck on how to pull this all together in the task.

With all that said, how do I go about inserting these values into this list? The value for “zone” needs to come from the same file that the records themselves come from, and the value for “server” comes from the “ip_address” property that goes along with the inventory item.

I investigated set_fact, but couldn’t figure out how to structure that, and I’m not sure it would work to begin with since the “server” property is global to all zones and the “zone” property is specific to the zone being updated.

I’m not sure what to do to solve this, and I’m hopeful I’m just missing some fancy syntax somewhere.

Any thoughts or wisdom are greatly appreciated

4 posts - 3 participants