Issue creating a basic ansible dynamic inventory

I’m trying to create a basic dynamic inventory script for ansible based on json output. I’m new to jq but I’ve hit an issue where the dynamic script on ansible v2.9.14 & 2.9.15 doesn’t like the output, but if I send the output to a file and then run Ansible against the output in the file, ansible works.

I know there will be syntax issues with trailing commas on the last key:value and I have an awful sed at the end of the curl to get it to look right.

This is what happens:

dynamic inventory script output:

{
"all": {
"hosts": {
"ip-172-31-39-30.eu-west-1.compute.internal": null,
"ip-172-31-44-224.eu-west-1.compute.internal": null,
"ip-172-31-42-6.eu-west-1.compute.internal": null,
"ip-172-31-32-68.eu-west-1.compute.internal": null,
}
}
}

Ansible run and error:

$ ansible -i ./dynamic1.sh all -m ping -u ubuntu
[WARNING]: * Failed to parse /home/ubuntu/dynamic1.sh with script plugin: failed to parse executable inventory script results from /home/ubuntu/dynamic1.sh:
Expecting property name enclosed in double quotes: line 8 column 5 (char 242)
[WARNING]: * Failed to parse /home/ubuntu/dynamic1.sh with ini plugin: /home/ubuntu/dynamic1.sh:2: Expected key=value host variable assignment, got: {
[WARNING]: Unable to parse /home/ubuntu/dynamic1.sh as an inventory source
[WARNING]: No inventory was parsed, only implicit localhost is available
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'

Now, if I output the dynamic script run to a file, then run ansible again, it works:

$ ./dynamic1.sh > output.json

$ cat output.json
{
"all": {
"hosts": {
"ip-172-31-39-30.eu-west-1.compute.internal": null,
"ip-172-31-44-224.eu-west-1.compute.internal": null,
"ip-172-31-42-6.eu-west-1.compute.internal": null,
"ip-172-31-32-68.eu-west-1.compute.internal": null,
}
}
}

$ ansible -i output.json all -m ping -u ubuntu
[DEPRECATION WARNING]: Distribution Ubuntu 16.04 on host ip-172-31-42-6.eu-west-1.compute.internal should use /usr/bin/python3, but is using /usr/bin/python for
backward compatibility with prior Ansible releases. A future Ansible release will default to using the discovered platform python for this host. See
https://docs.ansible.com/ansible/2.9/reference_appendices/interpreter_discovery.html for more information. This feature will be removed in version 2.12. Deprecation
warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.
ip-172-31-42-6.eu-west-1.compute.internal | SUCCESS => {
"ansible_facts": {
"discovered_interpreter_python": "/usr/bin/python"
},
"changed": false,
"ping": "pong"
}
ip-172-31-39-30.eu-west-1.compute.internal | SUCCESS => {
"ansible_facts": {
"discovered_interpreter_python": "/usr/bin/python3"
},
"changed": false,
"ping": "pong"
}
ip-172-31-32-68.eu-west-1.compute.internal | SUCCESS => {
"ansible_facts": {
"discovered_interpreter_python": "/usr/bin/python3"
},
"changed": false,
"ping": "pong"
}
ip-172-31-44-224.eu-west-1.compute.internal | SUCCESS => {
"ansible_facts": {
"discovered_interpreter_python": "/usr/bin/python3"
},
"changed": false,
"ping": "pong"
}

This is the contents of dynamic1.sh. I know there will be better ways to do this but I just need a list of servers based on a matching variable in the JSON output.

$ cat dynamic1.sh
#!/bin/bash
echo "{"
echo " \"all\": {"
echo " \"hosts\": {"
curl --silent -X GET https://url.com/api/servers -H "Authorization: Token $token" -H "Content-Type: text/json" -H "Accept:application/json" | jq -r '.Result.servers[] | select(.ansible_local.local.local_facts.instance_type | tostring | contains("t2.micro")) | (.ansible_fqdn+"\": null,")' | sed 's/^/"/g'
echo " }"
echo " }"
echo "}"

Can anyone give me any help on why does ansible accepts the file but not the output of the script?

It's wrong JSON syntax. Remove the comma at the end of line 7.

Thanks Vladimir,

When I take out the trailing comma on line 7 I get a different error:

$ ./dynamic1.sh
{
“all”: {
“hosts”: {
“ip-172-31-39-30.eu-west-1.compute.internal”: null,
“ip-172-31-44-224.eu-west-1.compute.internal”: null,
“ip-172-31-42-6.eu-west-1.compute.internal”: null,
“ip-172-31-32-68.eu-west-1.compute.internal”: null
}
}
}
$ ansible -i ./dynamic1.sh all -m ping -u ubuntu
[WARNING]: * Failed to parse /home/ubuntu/dynamic1.sh with script plugin: You defined a group ‘all’ with bad data for the host list: {u’hosts’: {u’ip-172-31-32-68
.eu-west-1.compute.internal’: None, u’ip-172-31-39-30.eu-west-1.compute.internal’: None, u’ip-172-31-42-6.eu-west-1.compute.internal’: None, u’ip-172-31-44-224.eu-
west-1.compute.internal’: None}}
[WARNING]: * Failed to parse /home/ubuntu/dynamic1.sh with ini plugin: /home/ubuntu/dynamic1.sh:3: Expected key=value host variable assignment, got: {
[WARNING]: Unable to parse /home/ubuntu/dynamic1.sh as an inventory source
[WARNING]: No inventory was parsed, only implicit localhost is available
{ | UNREACHABLE! => {
“changed”: false,
“msg”: “Failed to connect to the host via ssh: ssh: Could not resolve hostname {: Temporary failure in name resolution”,
“unreachable”: true
}

Ansible still doesn’t accept the dynamic script’s output. Ansible cli will accept the file output being used with or without the last “,” so it’s odd for me. I’m supplying the list of hosts in a format that Ansible will run against based on the file (output of the script), but ansible won’t take the same output from the script.

Thanks Vladimir,

When I take out the trailing comma on line 7 I get a different error:

$ ./dynamic1.sh
{
"all": {
"hosts": {
"ip-172-31-39-30.eu-west-1.compute.internal": null,
"ip-172-31-44-224.eu-west-1.compute.internal": null,
"ip-172-31-42-6.eu-west-1.compute.internal": null,
"ip-172-31-32-68.eu-west-1.compute.internal": null
}
}
}

hosts should be a list, not a dict, e.g.

{
    "group": {
        "hosts": [
            "192.168.28.71",
            "192.168.28.72"
        ],
        "vars": {
            "ansible_ssh_user": "johndoe",
            "ansible_ssh_private_key_file": "~/.ssh/mykey",
            "example_variable": "value"
        }
    },
    "_meta": {
        "hostvars": {
            "192.168.28.71": {
                "host_specific_var": "bar"
            },
            "192.168.28.72": {
                "host_specific_var": "foo"
            }
        }
    }
}

Example from: https://www.jeffgeerling.com/blog/creating-custom-dynamic-inventories-ansible

Regards
        Racke

Hey Racke,

I’ve changed the output to a list now so we can put the json format to bed hopefully now:

$ ansible --version
ansible 2.9.15
config file = /home/ubuntu/ansible.cfg
configured module search path = [u’/home/ubuntu/.ansible/plugins/modules’, u’/usr/share/ansible/plugins/modules’]
ansible python module location = /usr/lib/python2.7/dist-packages/ansible
executable location = /usr/bin/ansible
python version = 2.7.17 (default, Sep 30 2020, 13:38:04) [GCC 7.5.0]

$ ./dynamic3.sh
{
“all”: {
“hosts”: [
ip-172-31-10-247.eu-west-1.compute.internal,
] }
}

on ansible version - v2.9.15 I get this output:

$ ansible -i dynamic3.sh all -m ping -u ubuntu
[WARNING]: * Failed to parse /home/ubuntu/dynamic3.sh with script plugin: failed to parse executable inventory script results from /home/ubuntu/dynamic3.sh: No JSON
object could be decoded
[WARNING]: * Failed to parse /home/ubuntu/dynamic3.sh with ini plugin: /home/ubuntu/dynamic3.sh:2: Expected key=value host variable assignment, got: {
[WARNING]: Unable to parse /home/ubuntu/dynamic3.sh as an inventory source
[WARNING]: No inventory was parsed, only implicit localhost is available
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match ‘all’

But the same script on ansible version v2.9.6

$ ansible --version
ansible 2.9.6
config file = /home/ubuntu/ansible.cfg
configured module search path = [u’/home/ubuntu/.ansible/plugins/modules’, u’/usr/share/ansible/plugins/modules’]
ansible python module location = /usr/lib/python2.7/dist-packages/ansible
executable location = /usr/bin/ansible
python version = 2.7.17 (default, Sep 30 2020, 13:38:04) [GCC 7.5.0]

$ ./dynamic.sh
{
“all”: {
“hosts”: [
ip-172-31-10-247.eu-west-1.compute.internal,
] }
}

$ ansible -i dynamic.sh all -m ping -u ubuntu
[DEPRECATION WARNING]: Distribution Ubuntu 18.04 on host ip-172-31-10-247.eu-west-1.compute.internal should use
/usr/bin/python3, but is using /usr/bin/python for backward compatibility with prior Ansible releases. A future
Ansible release will default to using the discovered platform python for this host. See
https://docs.ansible.com/ansible/2.9/reference_appendices/interpreter_discovery.html for more information. This
feature will be removed in version 2.12. Deprecation warnings can be disabled by setting deprecation_warnings=False
in ansible.cfg.
ip-172-31-10-247.eu-west-1.compute.internal | SUCCESS => {
“ansible_facts”: {
“discovered_interpreter_python”: “/usr/bin/python”
},
“changed”: false,
“ping”: “pong”
}

and it works? Is anyone able to tell me the difference? You help would be greatly appreciated.

You're right. The inventory plugin script.py tests *hosts* is a list
https://github.com/ansible/ansible/blob/devel/lib/ansible/plugins/inventory/script.py#L182