Tweaking an inventory's groups

Get Help
,
1

I want to experiment with applying tweaks to an otherwise static inventory at runtime, but I’m not sure how to accomplish this. The existing inventory is a straightforward “ini” format single file, all hosts are in groups, and it contains no variables whatsoever. (None of that should matter, unless the answer turns out to be to make a custom inventory plugin from lib/ansible/plugins/inventory/ini.py, but that feels like a bad idea.)

By convention, all of our group names start with mw_ followed by a “service line” and an “environment” (examples: mw_splunk_tst, mw_sso_dev, mw_tableau_prd, mw_infoporte_spt) followed by as many additional qualifiers as needed per service line, concatenated with joining underscore characters.

While the vast majority of our group expressions target hosts within a particular environment (think mw_splunk_tst_ix_dc1 i.e. the test environment Splunk indexers in the data center #1), it’s occasionally convenient to address qualified hosts across all environments (like mw_splunk_all_ix_dc1).

It’s certainly possible to maintain such a static inventory by hand; we’ve done it for years. But it’s tedious; most of the inventory file is taken up defining the intermediate groups and their parents/children. With our group naming conventions, it should be possible to specify only the leaf groups – the ones actually containing hosts – and programmatically generate all the middle groups. And, given a list of allowed environments (dev, tst, spt, prd) we could generate the corresponding _all_ groups. From the example above, if the static inventory has a host-containing group mw_splunk_tst_ix_dc1, then we know we should have all these groups:

mw_splunk_tst_ix_dc1  mw_splunk_all_ix_dc1
mw_splunk_tst_ix      mw_splunk_all_ix
mw_splunk_tst         mw_splunk_all
mw_splunk

That would allow us to throw away most of our static inventory file while giving us consistent intermediate groups which have on occasion been inadvertently omitted, typo’d, etc. in the past.

So that’s the goal. But what are the reasonable options for achieving it? I’ve been bouncing all over the inventory plugin docs and sources without getting noticeably smarter.

2

ansible inventory file can actually be executable file - this is called ansible inventory script. Executable file should print out inventory in json format to stdout. See Working with dynamic inventory — Ansible Community Documentation

You can have your static inventory with all the host attributes in a file and a script file as you inventory that you pass to ansible-playbook -i <executable script> <playbook> (bash, python, any scripting language or executable file). This script will create json inventory from your manual representation and output it to stdout - so you will have inventory you need.
I think it is what you need - static inventory and a script that will transform it any way you like.

3

+1 for inventory script

The other solution is to use add_host module to add hosts to dynamically created groups. If I’m not mistaken, add_host will create groups that do not exist.

Because you have a nicely structured group names, you can split their names by underscore _ character, create a loop or jinja script that assembles all necessary groups and then call add_host. Something like:

- name: Add host to derived aggregate groups
  add_host:
    name: "{{ item }}"
    groups: |-
      {% set result = [] %}
      {% for g in hostvars[item]['group_names'] %}
        {% if g is match('^mw_[^_]+_[^_]+_[^_]+_[^_]+$') %}
          {% set p = g.split('_') %}
          {% set _ = result.append([
            p[0] ~ '_' ~ p[1] ~ '_' ~ p[2] ~ '_' ~ p[3],         # mw_splunk_tst_ix
            p[0] ~ '_' ~ p[1],                                   # mw_splunk
            p[0] ~ '_all_' ~ p[3] ~ '_' ~ p[4],                  # mw_splunk_all_ix_dc1
            p[0] ~ '_all_' ~ p[3],                               # mw_splunk_all_ix
            p[0] ~ '_all'                                        # mw_splunk_all
          ]) %}
        {% endif %}
      {% endfor %}
      {{ result | unique }}
  loop: "{{ ansible_play_hosts }}"

You need to call this task in a first play of the playbook. The second play then can be the regular play. The reason is that inventory is not refreshed during the same play but is ready to be used for the next one.

Jinja block is courtesy of ChatGPT… I was lazy :disappointed_face:. The code conceptually looks OK but not tested. ChatGPT was wrong for one thing, though. It’s code was missing the loop. add_host is kinda special because it always runs only for a single host so for adding all the hosts to proper groups, looping trough the list of all hosts is required.

4

My first thought was a role, similar to what @bvitnik suggested. Instead of one task, I thought to have a few smaller tasks to parse the groups out. Since it’s all local processing it should be quick.

But I personally wouldn’t want to update all of my playbooks, and what your really asking for is like an inventory post processing. You can achieve that with the builtin constructed inventory plugin. That would allow you to leverage compose, keyed groups, etc and apply that on top of your ini inventory.

To do this, you would create a constructed yaml config and call your Ansible commands with two inventory files (-i foo.ini -i constructed.yml), or specify the directory and Ansible includes all inventory files it finds. Definitely an interesting problem!

5

ansible-inventory takes an inventory as an input and by default outputs as an ‘inventory script’ so it is a good way to test, also I would look at the constructed and generator inventory plugins

2 Likes