Changing the dictionary key names but not values of items in a loop

I have a role for Adguard Home with a task to add DHCP reservations. That task requires a list variable where each item has the same dictionary keys:

- name: Add static leases on AdGuard Home
  ansible.builtin.uri:
    method: POST
    return_content: true
    url: 'http://192.168.1.1/control/dhcp/add_static_lease'
    body_format: json
    body: |
      {
        "mac": "{{ item.mac }}",
        "ip": "{{ item.ip }}",
        "hostname": "{{ item.hostname }}"
      }
    url_username: "{{ adguard_home_user.name }}"
    url_password: "{{ adguard_home_user.password }}"
    force_basic_auth: true
  loop: "{{ static_leases }}"

This information is already stored in another list variable with different names for the dictionary keys while being used by other roles/playbooks.

dhcp_reservations:
  - mac_address: 52:54:00:1a:2b:3c
    number: 192.168.1.11
    name: laptop
  - mac_address: 52:54:00:4d:5e:6f
    number: 192.168.1.12
    name: desktop

I wish I could just put static_leases: "{{ dhcp_reservations }}" in the host’s variables and call it a day, but that won’t work obviously.
I’m not at liberty to alter or edit one of these roles, so I have at least one case where I can’t change the task that uses these variables to fit the dictionary keys being used already.

How can I/should I transform the name of these dictionary keys on each item in the list while preserving the value? Alternatively, how can I add a new dictionary key with the needed name and the same value?

# Example desired outcome
dhcp_reservations:
  - mac: 52:54:00:1a:2b:3c
    ip: 192.168.1.11
    hostnamename: laptop
  - mac: 52:54:00:4d:5e:6f
    ip: 192.168.1.12
    hostname: desktop

# Example alternative desired outcome
dhcp_reservations:
  - mac: 52:54:00:1a:2b:3c
    ip: 192.168.1.11
    hostnamename: laptop
    mac_address: 52:54:00:1a:2b:3c
    number: 192.168.1.11
    name: laptop
  - mac: 52:54:00:4d:5e:6f
    ip: 192.168.1.12
    hostname: desktop
    mac_address: 52:54:00:4d:5e:6f
    number: 192.168.1.12
    name: desktop

I tried going through Ansible documentation, filters, etc, but couldn’t seem to find a solution that worked with dictionary keys on list items.
I apologize for not giving more concrete examples of what I tried, but I haven’t touched it over the weekend, and it became all a blur.
Can’t shake the feeling that the answer is simple, and I’m just overlooking something obvious.
I appreciate any solutions people might have.

Someone brighter than me might be able to do this without a loop, perhaps using map or json_query / JMESPath or something but I’d do it the slow and simple way with a loop through the original list to create a new list, something like this (untested!)

- name: Set a fact for the AdGuard Home static leases
  ansible.builtin.set_fact:
    adguard_list: "{{ adguard_list | default([]) + adguard_item }}"
  vars:
    adguard_item: |
      {
        "mac": "{{ dhcp_reservation.mac_address }}",
        "ip": "{{ dhcp_reservation.number }}",
        "hostname": "{{ dhcp_reservation.name }}"
      }
  loop: dhcp_reservations
  loop_control:
    loop_var: dhcp_reservation

- name: Print the AdGuard Home static leases list
  ansible.builtin.debug:
    var: adguard_list

I have definitely tried a few things involving map().

I was hoping that I wouldn’t have to run some pre_tasks in the playbook just to get this to work, but it is a good simplistic interim solution.
I’ll keep it in mind for sure.

Try using JMESPath (which needs to be installed on the Ansible controller), I think this will work,

- name: Set a fact for the AdGuard Home static leases
  ansible.builtin.set_fact:
    adguard_list: "{{ dhcp_reservations | community.general.json_query('[].{mac: mac_address, ip: number, hostname: name}') }}"

- name: Print the AdGuard Home static leases list
  ansible.builtin.debug:
    var: adguard_list

You could skip the fact setting step if you want and use the above as the body for the uri module.

I find the JMESPath Terminal really handy for working out queries like this:

2 Likes

I marked it as the solution because it does work perfectly for the issue as I described it. Thank you @chris!

Unfortunately, I may have shot myself in the foot by simplifying my example variable too much. In my inventory vars, it actually looks like this:

dhcp_reservations:
  - mac_address: 52:54:00:1a:2b:3c
    number: 11
    name: laptop
  - mac_address: 52:54:00:4d:5e:6f
    number: 12
    name: desktop

Each of the number dictionary keys also needs its value transformed/filtered into an actual IP address, but that is a separate and different problem I believe.

1 Like

Post some more details if you want, I’d hope that someone here could help!

That Jinja is so broken ill-designed that it can’t do things out-of-the box that are so easily done by jmespath, jq, etc. is disappointing. But what truly makes me sad is that Ansible (which I care about a lot) is saddled with Jinja (which I have no need for outside of Ansible).

I understand how this came to be. Incorporating Jinja must have saved a lot of development effort when Ansible was in its formative years. And strictly as a templating tool Jinja is arguably adequate.

But Ansible and its users deserve a data expression language worthy of the challenges that Jinja studiously avoids rising to. I can envision a drop-in replacement that handles all current valid Jinja expressions, but that integrates better with Ansible (fully qualified variables anyone?) while not shying away from real power.

The first target on my list would be the “map” filter. In addition to the currently accepted special case keywords and its fixed list of filters (which ironically includes “map” itself) , it should also accept arbitrarily complex expressions. “map” is supposed to be the pipeline’s answer to loops, but why limit what you can do in those loops to a short list of anticipated operations? The OP’s problem could be solved without having to resort to outside libraries if “map” weren’t thus hobbled.

I’m not ranting at you, @chris; I’m just generically ranting. It landed here because I wasted a couple of hours trying to solve this problem using built-in tools. It’s not the first time I’ve set off on this fool’s errand either, so I should have known better. Should not have to pull in extra libraries. Or create a custom filter in python. This should be possible, nay practical, to solve this using the built-in data manipulation language. But we’re stuck with Jinja.

— “I’m looking at you, Jinja! I’m calling you out!”

2 Likes

Full points for trying!

The JMESPath answer seems fairly efficient to me :person_shrugging: :

I’ve found JMESPath easier to learn than Jinja because of the associated tools, jp and jpterm and the fact that it’s an extra library doesn’t really bother me…

In fact I like the fact that Ansible can use other libraries like JMESPath, another favourite of mine is JC which can be used via the community.general.jc filter.

I would however be very interested in seeing your ideas and suggestions being worked on and developed further @utoddl :slight_smile: .

2 Likes

Looks like we’re all dummies. Was perusing Ansible documentation’s Index of all Filter Plugins and found ansible.utils.replace_keys (direct documentation link).

Given my example variable reproduced here:

dhcp_reservations:
  - mac_address: 52:54:00:1a:2b:3c
    number: 192.168.1.11
    name: laptop
  - mac_address: 52:54:00:4d:5e:6f
    number: 192.168.1.12
    name: desktop

All I had to do is pass it through the filter like so:
"{{ dhcp_reservations | ansible.utils.replace_keys(target=[{'before': 'mac_address', 'after': 'mac'}, {'before': 'number', 'after': 'ip'}, {'before': 'name', 'after': 'hostname'}]) }}"

And the desired result is achieved.
grumbles about having looked over that list of filters a dozen times over

3 Likes

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.