AWX inventory source from python script

Hi everyone!
I’m facing an issue that I’m unable to solve.
I’ve a AWX installation, and I’m trying to populate an inventory with a script. It’s just an api call to Linode to list my instances, and then I present the output like ansible inventory wants it (or at least, what I think it wants).
In my local pc, if I run

ansible-inventory --inventory ./linode_inventory.py --list

I can see my list no problem. The output looks like this:

{
    "_meta": {
        "hostvars": {
            "instance.example1": {
                "id": 1111111,
                "private_ip": null,
                "public_ip": "11.11.11.11",
                "region": "us-east",
                "status": "running",
                "tags": [
                ]
            },
            "instance.example2": {
                "id": 2222222,
                "private_ip": null,
                "public_ip": "22.22.22.22",
                "region": "us-east",
                "status": "running",
                "tags": [
                ]
            }
        }
    },
    "all": {
        "children": [
            "ungrouped"
        ]
    },
    "ungrouped": {
        "hosts": [
            "instance.example1",
            "instance.example2"
        ]
    }
}

So, what I did, is adding a Project on AWX with this github repo, created an inventory, and as “source inventory” I selected “source from project”. When I try to sync the inventory though, I get:

ansible-inventory [core 2.15.10]
  config file = /runner/project/ansible.cfg
  configured module search path = ['/runner/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.9/site-packages/ansible
  ansible collection location = /runner/requirements_collections:/runner/.ansible/collections:/usr/share/ansible/collections:/usr/share/automation-controller/collections
  executable location = /usr/local/bin/ansible-inventory
  python version = 3.9.19 (main, Aug 23 2024, 00:00:00) [GCC 11.5.0 20240719 (Red Hat 11.5.0-2)] (/usr/bin/python3)
  jinja version = 3.1.4
  libyaml = True
Using /runner/project/ansible.cfg as config file
setting up inventory plugins
Loading collection ansible.builtin from 
host_list declined parsing /runner/project/inventory/linode_inventory.py as it did not pass its verify_file() method
auto declined parsing /runner/project/inventory/linode_inventory.py as it did not pass its verify_file() method
yaml declined parsing /runner/project/inventory/linode_inventory.py as it did not pass its verify_file() method
toml declined parsing /runner/project/inventory/linode_inventory.py as it did not pass its verify_file() method
[WARNING]:  * Failed to parse /runner/project/inventory/linode_inventory.py
with script plugin: failed to parse executable inventory script results from
/runner/project/inventory/linode_inventory.py: Expecting value: line 1 column 1
(char 0). Expecting value: line 1 column 1 (char 0)
  File "/usr/local/lib/python3.9/site-packages/ansible/inventory/manager.py", line 293, in parse_source
    plugin.parse(self._inventory, self._loader, source, cache=cache)
  File "/usr/local/lib/python3.9/site-packages/ansible/plugins/inventory/script.py", line 150, in parse
    raise AnsibleParserError(to_native(e))
[WARNING]:  * Failed to parse /runner/project/inventory/linode_inventory.py
with ini plugin: /runner/project/inventory/linode_inventory.py:2: Expected
key=value host variable assignment, got: requests
  File "/usr/local/lib/python3.9/site-packages/ansible/inventory/manager.py", line 293, in parse_source
    plugin.parse(self._inventory, self._loader, source, cache=cache)
  File "/usr/local/lib/python3.9/site-packages/ansible/plugins/inventory/ini.py", line 137, in parse
    raise AnsibleParserError(e)
[WARNING]: Unable to parse /runner/project/inventory/linode_inventory.py as an
inventory source
ERROR! No inventory was parsed, please check your configuration and options.

I get a similar error on my local pc if I try to run ansible-inventory against a yml inventory with this content:

plugin: ansible.builtin.script
script: ../inventory/linode_inventory.py

This is the python script. the value for the linode token it’s provided via awx credential attached to the inventory source:

#!/usr/bin/env python
import requests
import json
import os

# Set Linode API token (from environment variables for security)
LINODE_API_TOKEN = os.getenv('a_variable_coming_from_my_awx_credentials')

# Linode API base URL
API_URL = "https://api.linode.com/v4/linode/instances"

# Headers for authentication
headers = {
    "Authorization": f"Bearer {LINODE_API_TOKEN}",
    "Content-Type": "application/json"
}

def get_linode_instances():
    try:
        response = requests.get(API_URL, headers=headers)
        response.raise_for_status()
        return response.json()
    except requests.exceptions.RequestException as e:
        print(f"Error fetching Linode instances: {e}")
        return {}

def parse_linode_instances(data):
    hosts = []
    linode_instances = data.get('data', [])
    
    for linode in linode_instances:
        host = {
            'name': linode['label'],  # Instance name
            'public_ip': linode['ipv4'][0],  # Public IP address
            'private_ip': linode['ipv4'][1] if len(linode['ipv4']) > 1 else None,
            'region': linode['region'],
            'tags': linode['tags'],
            'status': linode['status'],
            'id': linode['id']
        }
        hosts.append(host)
    
    return hosts

def generate_inventory():
    linode_data = get_linode_instances()
    if not linode_data:
        print("No data received from Linode API")
        return  # Exit if no data is received

    instances = parse_linode_instances(linode_data)
    if not instances:
        print("No instances found.")
    
    # Prepare the inventory structure
    inventory = {
        '_meta': {
            'hostvars': {host['name']: {'public_ip': host['public_ip'], 'private_ip': host['private_ip'], 'region': host['region'], 'tags': host['tags'], 'status': host['status'], 'id': host['id']} for host in instances}
        },
        'all': {
            'hosts': [host['name'] for host in instances],
            'children': {}
        }
    }

    # Print the inventory for debugging
    print(json.dumps(inventory, indent=2))

if __name__ == '__main__':
    generate_inventory()

Any idea why?
Tks!
Cheers

Andrea

You have to turn this into an inventory plugin, with a small config file.

• Create a git repository with the inventory plugin configured correctly and also setup in the ansible.cfg that should reside in the top directory of the source tree
• Setup a project in AAP that uses the dynamic inventory plugin and configures it correctly
• Sync the project
• Create a new, empty inventory
• Setup the source to point to inventory project and as inventory file use the configuration file for the plugin. This file will probably not show up in the pull-down list, but it is possible to just insert the name of the file into the field.
• Sync the inventory

In the ansible.cfg something like:

[defaults]
inventory_plugins = plugins/inventory

[inventory]
enable_plugins = myscript

And in the plugin config file

---
plugin: myscript
1 Like

You do not need to convert it into an inventory plugin (though this has other benefits), but it seems that when executing from AWX the script is not returning valid JSON output (that is what the message means), its expecting { as the first character of the output, but it is not getting that.

1 Like

Thank you @tonk ! You’re a life saver!
It worked! Now I just need to solve one last problem. I’m passing the LINODE_API_TOKEN via awx Credential. I’m already using successfully those kind of credentials inside job templates, and the source inventory has a field for the Credentials
The custom credential looks like this one:

Input configuration
  - id: linode_api_token
    type: string
    label: Linode Token
    secret: true
Injector configuration
extra_vars:
  LINODE_API_TOKEN_1: '{{ linode_api_token }}'

I can just use it inside a playbook, but in this case, the python script doesn’t seem too be able to read it, using

LINODE_API_TOKEN = os.getenv('LINODE_API_TOKEN_1')

Any idea why? :pray:

Solved! In the Injector configuration I needed to use “env” instead of “extra_vars”!
Thank you again y’all :smiling_face_with_three_hearts: