How to merge several dicts

kks · February 23, 2026, 8:59am

This is continuation from discussion started with @utoddl in topic Cannot change a variable passed to a role from within a role - request for docs - #11 by utoddl

The premise is following - there is a role that takes a dictionary as one of the parameters. And with dictionary there is not merging of parameters that are set on different levels - preceding values just replace values with lower precedence level as per Using variables — Ansible Community Documentation

Question is how to merge configuration dict that is passed to the role from dicts that are set on different levels. Here is a list of levels I came up with (from least important to most important)

role defaults
host group
host

There might be several host groups - for instance group that contains the host, group that contains that group, group that contains that group, etc. That complication is not needed for example I would like to present, it can be easily extended.

@utoddl proposed following code

- name: Populate splunk inputs.conf
  become: true
  become_user: splunk
  ansible.builtin.template:
    src: '{{ splunk_forwarder_inputsconf }}'
    dest: '{{ splunk_forwarder_homedir }}/etc/system/local/inputs.conf'
    owner: splunk
    group: splunk
    mode: 0644
  notify: restart splunkforwarder
  vars:
    # 'conf_var_names' contains a list of all the variables starting with
    # "splunk_forwarder_inputs_conf_" except "…defaults".
    conf_var_names: "{{ [ [lookup('ansible.builtin.varnames',
                                 '^splunk_forwarder_inputs_conf_(?!defaults$).*', default='')]
                        ] | flatten
                        | map('split', ',')
                        | flatten }}"
    # Combine the 'splunk_forwarder_inputs_conf_defaults' role default
    # and any host- or group-specific inputs_conf variables that might
    # apply to the current play host.
    conf_var_values: "{{ splunk_forwarder_inputs_conf_defaults
                        | combine(lookup('ansible.builtin.vars', *conf_var_names)) }}"

I see that there are few improvements that can be made to that code. Here are the issues I would like to solve

Lookup `lookup(‘ansible.builtin.varnames’,‘^splunk_forwarder_inputs_conf_(?!defaults$).*’, default=‘’ does not guarantee order in which variable names will be returned, so when we combine them with combine(lookup('ansible.builtin.vars', *conf_var_names)) result might differ due to order being different
Lookup `lookup(‘ansible.builtin.varnames’,‘^splunk_forwarder_inputs_conf_(?!defaults$).*’, default=‘’ return all the variables which name starts with splunk_forwarder_inputs_conf_ so variable named splunk_forwarder_inputs_conf_whatever will be found and merged again providing incosistent results

I came up with the following code

ocalhost
  gather_facts: false
  vars:
    configuration_variable: "splunk_forwarder_inputs_conf_"
    suffixes:
      - default
      - host_group
      - host
    configuration_variable_all_existing_variables: "{{ [lookup('ansible.builtin.varnames', '^'~configuration_variable~'.*', default='')] | map('split',',') | flatten }}"
    configuration_variable_list_of_variable_names_with_suffixes: "{{ suffixes | map('regex_replace', '^', configuration_variable) }}"
    configuration_variable_list_of_variable_names_only_existing: "{{ configuration_variable_list_of_variable_names_with_suffixes | select('in', configuration_variable_all_existing_variables) }}"
    configuration_variable_dict: "{{ {} | combine(lookup('ansible.builtin.vars', *configuration_variable_list_of_variable_names_only_existing)) }}"
    splunk_forwarder_inputs_conf_default:
      option1: default
      option2: default
      option3: default
    splunk_forwarder_inputs_conf_host:
      option2: host
    splunk_forwarder_inputs_conf_host_group:
      option2: host_group
      option3: host_group
    splunk_forwarder_inputs_conf_doesnot_do_anything:
      option1: fail
      option2: fail
      option3: fail

  tasks:
    - name: Print all existing variable with name as prefix
      ansible.builtin.debug:
        var: configuration_variable_all_existing_variables

    - name: Print all suffixed variables
      ansible.builtin.debug:
        var: configuration_variable_list_of_variable_names_with_suffixes

    - name: Print configuration variable resulting dict
      ansible.builtin.debug:
        var: configuration_variable_dict

    - name: Print configuration variable resulting dict, but the order is different
      ansible.builtin.debug:
        var: configuration_variable_dict
      vars:
        suffixes:
          - default
          - host
          -

Ideas is the same as in the code from @utoddl, but here we control what suffixed will be used to construct dictionaries that will be merged - this is variable suffixes - order is very important in this list.

From suffixes list all the variables that can influence the result are condsructed in variable configuration_variable_list_of_variable_names_with_suffixes

Only existing variables when selected for combining - configuration_variable_list_of_variable_names_only_existing - in this list there are variables that can influence the result.

And when all the variables are combined in the order of the suffix list, I start combining from empty dict, default goes first and so on.

This works starting with ansible-core 2.13

uv run --with ansible-core==2.13 ansible-playbook merge_with_list.yml

bash-5.2$ uv run --with ansible-core==2.13 ansible-playbook merge_with_list.yml
[WARNING]: No inventory was parsed, only implicit localhost is available
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match ‘all’

PLAY [localhost] ********************************************************************************************************************************************************************************************************************************************

TASK [Print all existing variable with name as prefix] ******************************************************************************************************************************************************************************************************
ok: [localhost] => {
“configuration_variable_all_existing_variables”: [
“splunk_forwarder_inputs_conf_default”,
“splunk_forwarder_inputs_conf_host”,
“splunk_forwarder_inputs_conf_host_group”,
“splunk_forwarder_inputs_conf_doesnot_do_anything”
]
}

TASK [Print all suffixed variables] *************************************************************************************************************************************************************************************************************************
ok: [localhost] => {
“configuration_variable_list_of_variable_names_with_suffixes”: [
“splunk_forwarder_inputs_conf_default”,
“splunk_forwarder_inputs_conf_host_group”,
“splunk_forwarder_inputs_conf_host”
]
}

TASK [Print configuration variable resulting dict] **********************************************************************************************************************************************************************************************************
ok: [localhost] => {
“configuration_variable_dict”: {
“option1”: “default”,
“option2”: “host”,
“option3”: “host_group”
}
}

TASK [Print configuration variable resulting dict, but the order is different] ******************************************************************************************************************************************************************************
ok: [localhost] => {
“configuration_variable_dict”: {
“option1”: “default”,
“option2”: “host_group”,
“option3”: “host_group”
}
}

PLAY RECAP **************************************************************************************************************************************************************************************************************************************************
localhost : ok=4 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0

uv run --with ansible-core==2.20 ansible-playbook merge_with_list.yml

bash-5.2$ uv run --with ansible-core==2.20 ansible-playbook merge_with_list.yml
[WARNING]: No inventory was parsed, only implicit localhost is available
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match ‘all’