Ansible-inventory -e, --extra-vars

j-oss2023 · February 11, 2024, 5:18am

Hi All

there is very little information, but correct me if i’m wrong the doc states that ansible-inventory has the extra vars option

From ansible-inventory doc:
(can’t paste the link as im a new user, can only paste 2 links )

-e, --extra-vars
    set additional variables as key=value or YAML/JSON, if filename prepend with @. This argument may be specified multiple times.

From ansible-playbook doc:

-e, --extra-vars
    set additional variables as key=value or YAML/JSON, if filename prepend with @. This argument may be specified multiple times.

There’s literally nothing in those two pages that would give you the impression that it wasn’t the same option and did the same thing…

So, you’d assume straight off the bat, that it is the same functionality as running ansible-playbook with the extra vars option (-e, --extra-vars)

So anyway, the thing is, I try to use it as so:
ansible-inventory -i inventory/dynamic/tt-per-customer-mysql.yml --graph -e customer="My Fav Customer"

The var isn’t being interpolated within the inventory file:

#   MySQL Dynamic Inventory to query hosts based on customer
#
#   Required columns: 
#   'inventory_group', 'inventory_hostname'
#
#   TLDR:
#   Besides the required columns, each column will be assigned 
#   as an ansible hostvar dynamically.

plugin: mysql-dynamic-inventory
db_host: mysql_host
db_user: ansible
db_pass: mysql_pass
db_name: icingadb
db_query: |
  SELECT 
    host as inventory_hostname, 
    address as ansbile_host, 
    'openwrt' as inventory_group 
  WHERE customer = "{{customer}}"

Using a custom mysql dynamic inventory plugin that i made, that works fine… just not with the extra var

mysql-dynamic-inventory/mysql-dynamic-inventory.py at main · j-oss2023/mysql-dynamic-inventory · GitHub

Troubleshooting:
I have created tried creating a new plugin option

      test:
        description: test
        required: true

And in the plugin python code i did:

print(self.get_option('test'))

then in the inventory file i just assigned the option like so:

plugin: mysql-dynamic-inventory
db_host: mysql_host
db_user: ansible
db_pass: mysql_pass
db_name: icingadb
test: "{{customer}}"

The output after running ansible-inventory with the -e customer=“My Fav Customer” was:

{{customer}}

So it isn’t interpolating the correctly, not sure how / what else to try - i heard a few times @bcoca mention that this is an option of the plugin:

github.com/ansible/ansible

ansible-inventory ignores extra-vars

opened 08:53PM - 23 Mar 22 UTC

closed 07:14PM - 24 Mar 22 UTC

jaydabi

support:core bug affects_2.12

### Summary The documentation of `ansible-inventory` ( https://docs.ansible.c…om/ansible/latest/cli/ansible-inventory.html ) states, that `-e` / `--extra-vars` parameter is supported, but it seems extra-vars are ignored. ### Issue Type Bug Report ### Component Name ansible-inventory ### Ansible Version ```console $ ansible --version ansible [core 2.12.0] config file = None configured module search path = ['/home/jonas/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules'] ansible python module location = /home/jonas/.local/lib/python3.8/site-packages/ansible ansible collection location = /home/jonas/.ansible/collections:/usr/share/ansible/collections executable location = /home/jonas/.local/bin/ansible python version = 3.8.10 (default, Nov 26 2021, 20:14:08) [GCC 9.3.0] jinja version = 3.0.2 libyaml = True ``` ### Configuration ```console $ ansible-config dump --only-changed ``` ### OS / Environment $ cat /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=20.04 DISTRIB_CODENAME=focal DISTRIB_DESCRIPTION="Ubuntu 20.04.3 LTS" ### Steps to Reproduce * Install collection `ansible-galaxy collection install community.general` * Create inventory file `inventory_proxmox.yml` ```yaml plugin: community.general.proxmox url: '{{ pve_host }}' user: '{{ pve_user }}' password: '{{ pve_password }}' validate_certs: '{{ pve_validate_certificate }}' ``` * Run `ansible-inventory --list -i inventory_proxmox.yml -e pve_host="https://your-proxmox.host:8006" -e pve_user="service-user@pve" -e pve_password=your-secret pve_validate_certificate=true` ### Expected Results Replaces variables/placeholders with values and returns host information. ### Actual Results Returns error, containing the variable/placeholder instead of the variables content. ```console [WARNING]: * Failed to parse /home/jonas/Repos/automation/inventory_proxmox.yml with auto plugin: Invalid URL '{{ pve_host }}/api2/json/access/ticket': No schema supplied. Perhaps you meant http://{{ pve_host }}/api2/json/access/ticket? [WARNING]: * Failed to parse /home/jonas/Repos/automation/inventory_proxmox.yml with yaml plugin: Plugin configuration YAML file, not YAML inventory [WARNING]: * Failed to parse /home/jonas/Repos/automation/inventory_proxmox.yml with ini plugin: Invalid host pattern 'plugin:' supplied, ending in ':' is not allowed, this character is reserved to provide a port. [WARNING]: Unable to parse /home/jonas/Repos/automation/inventory_proxmox.yml as an inventory source [WARNING]: No inventory was parsed, only implicit localhost is available { "_meta": { "hostvars": {} }, "all": { "children": [ "ungrouped" ] } } ``` ### Code of Conduct - [X] I agree to follow the Ansible Code of Conduct

So it seems like its doable, but how on earth… i can’t find any doc, i’ve tried as best i can to reverse engineer the inventory plugin: - ansible.builtin.constructed ( inspecting the python plugin code ansible/blob/devel/lib/ansible/plugins/inventory/constructed.py)

Which is claimed to support extra vars, but i can’t figure out if it’s missing completely or i’m just not able to figure it out… lack of skillz

Can anyone shine some light on this ?
Would be forever grateful

Thanks in advance, sorry for the long essay of a question just wanted to articulate it as best I could,

j-oss2023 · February 11, 2024, 7:03am

The plot thickens!!, i’m starting to think this is simply not possible in its current iteration…

So temporarily i made my plugin bare bones, removing the logic to connect to the DB and iterate through the returned rows, adding them as inventory.
Instead, the plugin just looks like this:

    def _fetch_hosts(self):
        '''Fetches hosts from the database and populates the inventory'''
        group_init = {}
        try:
            self.inventory.add_group("ignored_group")
            self.inventory.add_host("ignore", group="ignored_group")
            self.inventory.set_variable("ignore", "ignore", "ignore")
        except Exception as e:
            raise AnsibleError(f"Database query failed: {e}")
        finally:
            pass

ansible-inventory -i inventory/dynamic/tt-per-customer-mysql.yml --list -e customer_name="SECRET PTY LTD"

So i can see, that the extra var is being added to the _meta, albeit in a funny way… (bug?)

{
    "_meta": {
        "hostvars": {
            "ignore": {
                "_raw_params": "PTY LTD",
                "customer_name": "SECRET",
                "ignore": "ignore"
            }
        }
    },
    "all": {
        "children": [
            "ungrouped",
            "ignored_group"
        ]
    },
    "ignored_group": {
        "hosts": [
            "ignore"
        ]
    }
}

moreover … if i do this:

self.inventory.add_group("ignored_group")
self.inventory.add_host("ignore", group="ignored_group")
self.inventory.set_variable("ignore", "ignore", "ignore")
var = self.inventory.get_host("ignore").get_vars().get('inventory_hostname')
print(var)

It outputs:

ignore

But… if i try and access the so called ‘extra variable’ changing the key to:

customer_name = self.inventory.get_host("ignore").get_vars().get('customer_name')

It outputs:

None

OK… so it’s not accessible to me i give up… it’s not going to happen

But basically this is what i’m trying to do:

gist.github.com

https://gist.github.com/j-oss2023/99e140476aa00a6a9d1aafcd73aa1053

dilemma.md

To run an Ansible playbook against a massive inventory, limiting it only to hosts where a specific host variable meets a condition (in your case, `customer` equals `'Ferrari'`), without using the `when` conditional in tasks or plays, you need to dynamically create a subset of your inventory that matches this condition before executing your playbook. This can be done by filtering your inventory outside of Ansible or by using Ansible's dynamic inventory feature. Here’s how you can achieve it:

### 1. Using a Dynamic Inventory Script

You can write a custom dynamic inventory script that filters your static inventory based on the `customer` host variable. This script can be written in any language (Python is commonly used) and should output JSON in the format expected by Ansible.

Here’s a simplified approach:

1. **Read your existing inventory**: Parse your existing inventory file(s) to find hosts.
2. **Filter hosts by the `customer` variable**: Look for the hosts that have `customer` set to `'Ferrari'`.

This file has been truncated. show original

And it seems to be out of reach ? I dont believe it

gist.github.com

https://gist.github.com/j-oss2023/78becd6ee40be86220d86451303650ec

noway.md

Integrating dynamic inventory filtering with Ansible AWX/Tower requires a slightly different approach due to AWX/Tower's GUI and its way of managing inventories. To pass variables dynamically and use them in filtering within AWX/Tower, you'll leverage AWX/Tower's survey feature or use predefined variables in your job templates. Here’s a high-level approach to achieve this:

### Step 1: Create a Custom Dynamic Inventory Script

1. **Write a dynamic inventory script** that accepts an environment variable or a command-line argument to filter the hosts based on the `customer` attribute. For example, your script could read an environment variable (e.g., `CUSTOMER_FILTER`) to apply the filter.
   
2. **Ensure your script outputs JSON** formatted for Ansible, containing only the hosts that match the `customer` value you're interested in.

### Step 2: Use the Dynamic Inventory Script in AWX/Tower

This file has been truncated. show original

Why oh why …

I’m kidding, i’m not really giving up … stay tuned.

j-oss2023 · February 11, 2024, 9:07am

BOOM!

ansible.builtin.constructed inventory was my answer all along, thank you @bcoca for the hint!

That’s done it folks, OK for anyone that needs this information be clear,

ansible-inventory -e, --extra-vars

AFAIK Does two things:

Appends a hostvar, to each host in the inventory - Quite useless in that sense, I can’t think of why you’d want to do this, unless ----> go to 2.
Except for the (maybe) only use case - constructed inventory - in a funny twist… this is how it works, rather counterintuitively:

My goal was a simple one, yet getting there I had to (with the help of ChatGPT, who’s useless at best) - setup my project like so…

my-inv-mysql.yml:

plugin: mysql-dynamic-inventory
db_host: icingadb_host
db_user: ansible
db_pass: banana_man
db_name: icingadb
db_query: |
    SELECT 
        'openwrt' AS inventory_group,
        h.address AS ansible_host, 
        h.name AS inventory_hostname,
        MAX(CASE 
            WHEN s.name = 'FirmwareVersion' THEN REPLACE(REPLACE(REPLACE(ss.output, 'SNMP OK - FirmwareVersion', ''), '"', ''), '|', '') 
            END) AS firmware,
        MAX(CASE 
            WHEN cv1.name = 'customer' THEN SUBSTRING(cv1.value FROM 2 FOR CHAR_LENGTH(cv1.value) - 2)
            END) AS customer,
        MAX(CASE 
            WHEN cv2.name = 'location' THEN SUBSTRING(cv2.value FROM 2 FOR CHAR_LENGTH(cv2.value) - 2)
            END) AS location,
        MAX(CASE 
            WHEN s.name = 'MobileConnection' THEN REPLACE(REPLACE(REPLACE(ss_mobile.output, 'SNMP CRITICAL - status *', ''), '"', ''), '*', '') 
            END) AS operator
    FROM 
        host h
    INNER JOIN 
        host_customvar hcv1 ON h.id = hcv1.host_id
    INNER JOIN 
        customvar cv1 ON hcv1.customvar_id = cv1.id AND cv1.name = 'customer'
    INNER JOIN 
        host_customvar hcv2 ON h.id = hcv2.host_id
    INNER JOIN 
        customvar cv2 ON hcv2.customvar_id = cv2.id AND cv2.name = 'location'
    INNER JOIN 
        service s ON h.id = s.host_id
    INNER JOIN 
        service_state ss ON h.id = ss.host_id AND s.id = ss.service_id
    LEFT JOIN 
        service_state ss_mobile ON h.id = ss_mobile.host_id AND s.id = ss_mobile.service_id AND s.name = 'MobileConnection'
    WHERE 
        h.id IN (
            SELECT 
                DISTINCT h.id 
            FROM 
                host h
            INNER JOIN 
                service_state ss ON h.id = ss.host_id
            WHERE 
                (ss.output LIKE 'SNMP CRITICAL - status %disconnected%' OR ss.output LIKE 'SNMP CRITICAL - status %Disconnected%')
                AND h.id NOT IN (
                    SELECT 
                        h.id 
                    FROM 
                        host h
                    INNER JOIN 
                        service_state ss ON h.id = ss.host_id
                    WHERE 
                        ss.output LIKE 'SNMP CRITICAL - SimStatus %not inserted%'
                )
        )
        AND s.name IN ('MobileConnection', 'FirmwareVersion')
    GROUP BY
        h.id;

No changes were required to my mysql-dynamic-plugin

The trick is in the constructed inventory, which took a while to figure out i had to have the compose bit in there, before evaluating the groups,
without compose, it was complaining that customer_name variable didn’t exist, fair enough…

dynamic_customer_specific.yml:

---
plugin: constructed
strict: True
use_extra_vars: True
compose:
  customer_name: customer_name
groups:
  target_customer: customer == customer_name

And finally…

ansible-inventory -i my-inv-mysql.yml -i dynamic_customer_specific.yml -e 'customer_name="SECRET COMPANY PTY LTD"' playbooks/setup.yml -l target_customer

That’s how you make a dynamic, dynamic inventory based on an extra var … wow that was EASY!!!

N.B: Also my bad, about the -e it needs to be encapsulated in quotes like shown above, otherwise it splits it like I had assumed earlier was a bug… silly me…

Anyway, there must be a way to document this stuff ?
Save the next guy all this fun … happy to contribute , if anyone of the contributers could sprinkle their 2 cents and direct me on how to do so…

system · March 12, 2024, 9:07am

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.