Debugging constructed inventory || constructing group from values of host variables

Get Help
1

Hi all,

my goal is to create my hostgroups using the constructed plugin.
Most of it works well.
The only not working part is creating a group of hosts, that occur as value of a specific host variable.
Specifically Some of my hosts have the “jailhost” variable set to the name of another host and I want to create a “jailhosts” group, which contains all the hosts named in this variable of any host.

In this setting I have two questions for you.
First, what do I have to change in my example for it to work as I want?
Second, if you have the time for it, how do I debug such problems myself?
A minimal (not) working example of my 02groups.yml is:

plugin: ansible.builtin.constructed
strict: false
compose:
  jailhostnames: groups['all']|map('extract', hostvars, 'jailhost')|list|unique
keyed_groups:
  - prefix: jails
    key: jailhost
    parent_group: jails
groups:
  jailhosts: inventory_hostname in jailhostnames
  #jailhosts: inventory_hostname in (groups['jails']|map('extract', hostvars, 'jailhost')|unique)

A sample 01somejail.yml:

all:
  hosts:
    somejail:
      jailhost: vps1

Inspecting the resulting inventory with
ansible-inventory --list
shows, that the “jails” and “jails_” groups get created as expected, but there is no “jailhosts” group.
this does not change no matter which of the commented versions I use.
The filter expression itself works fine. printing its result in a debug task shows the correct list.

2

The all and ungrouped don’t get auto generated until ALL inventory sources are parsed, they would only exist for the constructed file if they had been defined in the static one. Also ‘composed’ varaibles are not available until the end of the plugin run, so you cannot define and use in another source. The groups variable is also unavailable, it gets created at Ansible runtime.

3

To debug i recommend setting ‘strict’ to `true’.

4

Thanks, I did not know that, but as shown in the static 01somejail.yml, I define the all group and as the filename starts with 01, it will be loaded before I use it in the 02groups.yml.
Setting strict to true does not change the behavior and produces no warnings.
If “groups” does not exist, how can I replace the groups[‘all’]?

5

with strict: true you should see additional text in existing warnings, in my case it showed jailhostnames templating failed due to undefined groups.

6

As for how to get what you want, i was looking at this patch

diff --git a/lib/ansible/plugins/inventory/constructed.py b/lib/ansible/plugins/inventory/constructed.py
index ee2b9b4295..2e83dd95cc 100644
--- a/lib/ansible/plugins/inventory/constructed.py
+++ b/lib/ansible/plugins/inventory/constructed.py
@@ -157,6 +157,8 @@ class InventoryModule(BaseInventoryPlugin, Constructable):
 
                 # get available variables to templar
                 hostvars = self.get_all_host_vars(inventory.hosts[host], loader, sources)
+                hostvars['groups'] = inventory.get_groups_dict()  # adds the 'groups' magic var
+
                 if host in fact_cache:  # adds facts if cache is active
                     hostvars = combine_vars(hostvars, fact_cache[host])

simpler than earlier version … but ‘hostvars’ is still undefined …

7

After creating a new project without ansible.conf and only one 01somejail.yml I finally got the warning you mentioned.

At least now I know why it did not work and can search for workarounds.
Is the patch you shared about to get merged into upstream, or is this just an idea for how one could make my code work?
In my mind running an userpatched Ansible screams “you are doing something wrong”, so I would prefer a vanilla solution.
Nevertheless you deserve the solution flag for debugging the problem.

Edit:
Constructed groups seem to be available directly after declaring them and can be used in the next group declaration, so replacing groups[‘jails’] with jails in the in my example commented line works.
I was quite sure I tried that.
Warning for future reader:
From my understanding unless Ansible optimizes this out, the resulting map expression will be reevaluated for every host, so this has a complexity of O(ord(all)*ord(jails)) ≈ O(n^2)

8

So what I’m going to merge is a documentation update constructed, let users know some limitations by bcoca · Pull Request #84510 · ansible/ansible · GitHub

The patch above is not something I’m currently considering, also it would not be enough, but a work around would be to change from doing a single a-inventory call to 2 (so inventory is reconciled and ‘all’ and ‘ungrouped’ are populated’)

something like this (but this won’t work due to verify_file)

ansible-inventory -i <$(ansible-inventory -i inv01.yml --list --export --yaml)  -i inv02.yml

So saving it as a yaml result or making the expression into a script you call (no yaml then).

This still requires a patch for ‘groups’ and ‘hostvars’ to be populated in the constructed plugin (or a local custom version of it).