Use dynamic inventory in AWX to maintain AWS Workspaces

Hello everyone,

I hope someone can help me with my problem.

My scenario:

We would like to regularly update our AWS workspaces (Linux) via AWX. However, as new hosts are regularly added and removed, I need a dynamic inventory. This is not a problem in itself, as it can now be created using a python script (GitHub - aws-samples/workspaces-with-ansible)

I have written the following python script, which creates an inventory from the AWS data:

#!/usr/bin/env python3

import argparse
import json
import os
import boto3
from botocore.config import Config

def generate_inventory(workspaces):
    inventory = {'_meta': {'hostvars': {}}}
    for ws in workspaces:
        state = ws.get('State', '')
        computer_name = ws.get('ComputerName', '')
        os = ws.get('WorkspaceProperties', {}).get('OperatingSystemName', '')
        workspace_id = ws.get('WorkspaceId', '')  # Added workspace_id
        username = ws.get('UserName', '')  # Added username

        if os == 'UBUNTU_22_04':
            group_name = 'ubuntu2204workspaces'
        elif os == 'AMAZON_LINUX_2':
            group_name = 'amazonlinux2workspaces'
    #    elif os == 'WINDOWS_SERVER_2016':
    #        group_name = 'windows_server_2016_workspaces'
    #    elif os == 'WINDOWS_SERVER_2019':
    #        group_name = 'windows_server_2019_workspaces'
    #    elif os == 'WINDOWS_SERVER_2022':
    #        group_name = 'windows_server_2022_workspaces'
    #    elif os == 'WINDOWS_10':
    #        group_name = 'windows_10_workspaces'
    #    elif os == 'WINDOWS_11':
    #        group_name = 'windows_11_workspaces'
        else:
            print(f"Warning: WorkSpace ({ws}) is running an operating system that is not supported by the dynamic inventory provider. See the GitHub page for support.")
            continue

        ip_address = ws.get('IpAddress', '')

        if state == 'AVAILABLE':
            group_name += 'available'
        elif state == 'STOPPED':
            group_name += 'stopped'
        else:
            group_name += f'_{state.lower()}'

        if group_name not in inventory:
            inventory[group_name] = {'hosts': [], 'vars': {}}

        inventory[group_name]['hosts'].append(ip_address)

        if group_name.startswith('windows'):
            host_vars = {
                'ansible_connection': 'winrm',
                'ansible_winrm_transport': 'kerberos',
                'ansible_winrm_port': '5985',
                'ansible_winrm_kerberos_hostname_override': computer_name,
                'workspace_id': workspace_id,  # Added workspace_id
                'username': username,  # Added username
                'state': state  # Added state
            }
        else:
            host_vars = {
                'ansible_connection': 'ssh',
                'ansible_user': 'ksxansible',
                'workspace_id': workspace_id,  # Added workspace_id
                'username': username,  # Added username
                'state': state  # Added state
            }

        inventory['_meta']['hostvars'][ip_address] = host_vars

    return inventory

def main():
    parser = argparse.ArgumentParser(description='Generate Ansible inventory for AWS WorkSpaces.')
    parser.add_argument('--region', help='AWS Region')
    parser.add_argument('--list', action='store_true', help='List inventory.')
    exclusivearguments = parser.add_mutually_exclusive_group()
    exclusivearguments.add_argument('--directory-id', help='DirectoryId.')
    exclusivearguments.add_argument('--workspace-ids', nargs='*', help='One or more WorkSpace IDs, separated by space.')
    args = parser.parse_args()

    if not args.list and not args.workspace_ids:
        parser.print_help()
        return

    region = args.region if args.region else os.environ.get('AWS_REGION')

    if not region:
        print('Error: AWS region must be specified via --region argument or AWS_REGION environment variable.')
        exit(1)

    workspaces_directory = args.directory_id if args.directory_id else os.environ.get('DIRECTORY_ID')

    specific_workspaces = args.workspace_ids

    config = Config(
        retries = {
            'max_attempts': 10,
            'mode': 'standard'
        }
    )

    client = boto3.client('workspaces', config=config, region_name=region)
    paginator = client.get_paginator("describe_workspaces")

    if specific_workspaces:
        operation_parameters= {
            "WorkspaceIds": specific_workspaces
            }
        workspaces = []
        for page in paginator.paginate(
            **operation_parameters, PaginationConfig={"PageSize": 25}
        ):
            workspaces.extend(page['Workspaces'])

    elif workspaces_directory:
        operation_parameters = {
            "DirectoryId": workspaces_directory
            }
        workspaces = []
        for page in paginator.paginate(
            **operation_parameters, PaginationConfig={"PageSize": 25}
        ):
            workspaces.extend(page['Workspaces'])
    else:
        workspaces = []
        for page in paginator.paginate(
            PaginationConfig={"PageSize": 25}
        ):
            workspaces.extend(page['Workspaces'])

    inventory = generate_inventory(workspaces)

    if args.list:
        print(json.dumps(inventory, indent=4))
    elif args.workspace_ids:
        print(json.dumps(inventory, indent=4))

if __name__ == '__main__':
    main()

before I can execute it, I export the required VARS:

export AWS_ACCESS_KEY_ID: 'AWS_ACCESS_KEY_ID'
export AWS_SECRET_ACCESS_KEY: 'AWS_SECRET_ACCESS_KEY'
export AWS_SESSION_TOKEN: 'AWS_SESSION_TOKEN'
export AWS_REGION: 'eu-central-1' 

This works fine when I run it from my local Linux Machine.

But now I want to run this dynamic inventory in AWX as well.

I have created a template and also used the python script as a source for my inventory. The question is: what do I do with the AWS keys?

I have already tried to specify them as an extra var in the template and I have also tried to create credentials with the keys, but unfortunately neither works. The inventory is not created and my playbook is empty

[WARNING]: Could not match supplied host pattern, ignoring:
ubuntu2204workspacesstopped
PLAY [Start AWS Workspaces (Ubuntu)] *******************************************
skipping: no hosts matched
[WARNING]: Could not match supplied host pattern, ignoring:
amazonlinux2workspacesstopped
PLAY [Start AWS Workspaces (Amazon Linux)] *************************************
skipping: no hosts matched
PLAY [localhost] ***************************************************************
TASK [Time to startPause for 60 seconds] ***************************************
Pausing for 60 seconds
(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)
ok: [localhost]
[WARNING]: Could not match supplied host pattern, ignoring:
ubuntu2204workspacesavailable
[WARNING]: Could not match supplied host pattern, ignoring:
amazon2workspacesavailable
PLAY [ubuntu2204workspacesavailable:amazon2workspacesavailable] ****************
skipping: no hosts matched
PLAY RECAP *********************************************************************
localhost                  : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0 

I’m getting a bit desperate. Does anyone have an idea how I could implement this?

The recommended way is to create a custom credential in AWX and attach that to your inventory source. Then access the credential’s exposed variables during your python script.
https://docs.ansible.com/ansible-tower/latest/html/userguide/credential_types.html

Another option is to directly expose the variables as environment variables by setting them in the source_vars in your inventory source. This isnt as secure as the custom credential, since the values will be in plain text.
https://docs.ansible.com/ansible-tower/latest/html/administration/scm-inv-source.html#scm-inventory-source-fields

Finally, you can set the environment variables for all of AWX. Go to Settings > Jobs settings > Edit and add them under Extra Environment Variables.
Technically any job can access the credentials at that point.

Thanks, it works with using the AWS credential template :slight_smile: