Need help on building ansible inventory file properly

Yehuda_Pinhas · December 30, 2019, 8:19am

Hi,
I have been searching all over the internet and there isn’t any good documentation that explains how to build my inventory properly. The official documentation is bad and there aren’t any example on inet that I could find.

Can you please help me? I need to build inventory file that contains 6 groups total and each group will have around 100 hosts in it…
this is what I currently have.

all:
hosts:
POC-ENV:
hosts:
ansible_host: /home/ansible/MACCABI/hosts/POC-ENV_list.yml
Nexus:
hosts:
ansible_host: /home/ansible/MACCABI/hosts/nexus_list.yml
Switches:
hosts:
ansible_host: /home/ansible/MACCABI/hosts/switch_list.yml
Avaya:
hosts:
ansible_host: /home/ansible/MACCABI/avaya_list.yml
Branch-Switches:
hosts:
ansible_host: /home/ansible/MACCABI/branch-switch_list.yml
Branch-Routers:
hosts:
ansible_host: /home/ansible/MACCABI/branch-router_list.yml

As you can see there are 6 groups: POC-ENV, Nexus, Switches, Avaya, Branch-Switches, Branch-Routers and I am trying to list all the hosts in a file and direct the inventory to read it from there since I have so many.

What am I doing wrong? can you fix my code?
Thanks for the helpers.

vbotka · December 30, 2019, 9:05am

It's possible to create the groups with "add_host"
https://docs.ansible.com/ansible/latest/modules/add_host_module.html

For example, create the dictionary "my_hosts" with all groups and hosts, and
"loop subelements"
https://docs.ansible.com/ansible/latest/plugins/lookup/subelements.html#subelements-traverse-nested-key-from-a-list-of-dictionaries

  - hosts: localhost
    vars:
      my_hosts:
        - group: 'POC_ENV'
          hosts: "{{ lookup('file',
                     '/home/ansible/MACCABI/hosts/POC-ENV_list.yml').splitlines() }}"
        - group: 'Nexus'
          hosts: "{{ lookup('file',
                     '/home/ansible/MACCABI/hosts/nexus_list.yml').splitlines() }}"
          ...
    tasks:
      - add_host:
          name: "{{ item.1 }}"
          groups: "{{ item.0.group }}"
        loop: "{{ lookup('subelements', my_hosts, 'hosts') }}"

(not tested)

Then use the groups in the next plays.

Notes:

* See "How to build your inventory"
  https://docs.ansible.com/ansible/latest/user_guide/intro_inventory.html#how-to-build-your-inventory
* 'POC-ENV', 'Branch-Switches' and 'Branch-Routers' are not valid names of
  the groups. See "Creating valid variable names"
  https://docs.ansible.com/ansible/latest/user_guide/playbooks_variables.html#creating-valid-variable-names
* See "Developing dynamic inventory"
  https://docs.ansible.com/ansible/latest/dev_guide/developing_inventory.html#developing-dynamic-inventory

Cheers,

-vlado

vbotka · December 30, 2019, 9:10am

Errata:

For example, create the list of dictionaries "my_hosts"

Yehuda_Pinhas · December 30, 2019, 9:31am

Hi again,

Thanks a lot for your reply.

I still don’t understand what these code lines used for in my case:

tasks:

add_host:
name: “{{ item.1 }}”
groups: “{{ item.0.group }}”
loop: “{{ lookup(‘subelements’, my_hosts, ‘hosts’) }}”

can you please explain?

I understand that you have wrote for me a lookup on the files mentioned for each lines which is exactly what I wanted. Now my intensions will be to copy and paste hosts of each environment to the appropriate host file.

Yehuda_Pinhas · December 30, 2019, 10:07am

Hi,

Can you please take a look on my errors and try to help me figuring out why I am getting them?

This is my inventory code:

hosts: localhost
vars:
my_hosts:
group: ‘POC_ENV’
hosts: “{{ lookup(‘file’,
‘/home/ansible/MACCABI/hosts/POC_ENV.yml’).splitlines() }}”

vars:
ansible_ssh_private_key_file: /home/ansible/.ssh/id_rsa
ansible_ssh_common_args: -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null
ansible_user: ansible

This is my playbook code (task is to add a VLAN):

name: Playing VLAN Configuration
hosts: POC-ENV
connection: local
vars:
vlan_id: 999
vlan_name: TEST_VLAN_TEST

tasks:

include_role:
name: add_vlan
name: show_vlan

This is the errors I get:

Thanks!

jean-yves · December 30, 2019, 10:20am

Hi,

It’s difficult to understand what you want to do… Perhaps you should explain in plain english before trying to code it ?

In your code, there’s a “hosts” directive in your inventory… You should not have it in this file (“hosts” directive is for playbooks)

If you have multiple inventory, perhaps a way to select which inventory to select is to add an argument to your running playbook

ansible-playbook -i /home/ansible/MACCABI/hosts/POC_ENV.yml myplaybook.yml

Regards,

vbotka · December 30, 2019, 10:41am

Short answer: These code lines create the groups you want.

Details:

1) Quoting from "add_host"
https://docs.ansible.com/ansible/latest/modules/add_host_module.html

"Use variables to create new hosts and groups in inventory for use in later
plays of the same playbook."

2) For example, with this list of dictionaries

    "my_hosts": [
        {
            "group": "POC_ENV",
            "hosts": [
                "POC_ENV_01",
                "POC_ENV_02",
                "POC_ENV_03"
            ]
        },
        {
            "group": "Nexus",
            "hosts": [
                "nexus_01",
                "nexus_02",
                "nexus_03"
            ]
        }
    ]

3) The debug task

      - debug:
          msg:
            - "name: {{ item.1 }}"
            - "groups: {{ item.0.group }}"
        loop: "{{ lookup('subelements', my_hosts, 'hosts') }}"

gives output that explains the plugin "subelements"

    "msg": [
        "name: POC_ENV_01",
        "groups: POC_ENV"
    ]
    "msg": [
        "name: POC_ENV_02",
        "groups: POC_ENV"
    ]
    "msg": [
        "name: POC_ENV_03",
        "groups: POC_ENV"
    ]
    "msg": [
        "name: nexus_01",
        "groups: Nexus"
    ]
    "msg": [
        "name: nexus_02",
        "groups: Nexus"
    ]
    "msg": [
        "name: nexus_03",
        "groups: Nexus"
    ]

4) The add_host tasks

      - add_host:
          name: "{{ item.1 }}"
          groups: "{{ item.0.group }}"
        loop: "{{ lookup('subelements', my_hosts, 'hosts') }}"

create groups that can be used in the next plays. For example the play

  - hosts: POC_ENV
    tasks:
      - debug:
          var: inventory_hostname

gives

  ok: [POC_ENV_01] => {
      "inventory_hostname": "POC_ENV_01"
  }
  ok: [POC_ENV_02] => {
      "inventory_hostname": "POC_ENV_02"
  }
  ok: [POC_ENV_03] => {
      "inventory_hostname": "POC_ENV_03"
  }

HTH,

-vlado

vbotka · December 30, 2019, 11:26am

Sure. As a working example, the playbook

  - hosts: localhost
    vars:

      my_hosts:
        - group: 'POC_ENV'
          hosts: "{{ lookup('file',
                     '/home/ansible/MACCABI/hosts/POC-ENV_list.yml').splitlines() }}"
    tasks:
      - add_host:
          name: "{{ item.1 }}"
          groups: "{{ item.0.group }}"
          ansible_ssh_private_key_file: /home/ansible/.ssh/id_rsa
          ansible_ssh_common_args: -o StrictHostKeyChecking=no
                                   -o UserKnownHostsFile=/dev/null
          ansible_user: ansible
        loop: "{{ lookup('subelements', my_hosts, 'hosts') }}"

  - hosts: POC_ENV
    tasks:
      - debug:
          msg: "{{ msg.split('\n') }}"
        vars:
          msg: |
            host [{{ inventory_hostname }}]
            key [{{ ansible_ssh_private_key_file}}]
            args [{{ ansible_ssh_common_args }}]
            user [{{ ansible_user }}]

gives

ok: [POC_ENV_01] => {
    "msg": [
        "host [POC_ENV_01]",
        "key [/home/ansible/.ssh/id_rsa]",
        "args [-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null]",
        "user [ansible]",
        ""
    ]
}
ok: [POC_ENV_02] => {
    "msg": [
        "host [POC_ENV_02]",
        "key [/home/ansible/.ssh/id_rsa]",
        "args [-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null]",
        "user [ansible]",
        ""
    ]
}
ok: [POC_ENV_03] => {
    "msg": [
        "host [POC_ENV_03]",
        "key [/home/ansible/.ssh/id_rsa]",
        "args [-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null]",
        "user [ansible]",
        ""
    ]
}

Notes:

* In your code "hosts: POC-ENV" the name of the group is not valid.
* See "add_host" examples how to add variables to the inventory
https://docs.ansible.com/ansible/latest/modules/add_host_module.html#examples
* Run "ansible-lint playbook.yml" before posting the code

HTH,

-vlado

vbotka · December 30, 2019, 11:55am

It's possible to put the variables into the "group_vars". For example

       - add_host:
           name: "{{ item.1 }}"
           groups: "{{ item.0.group }}"
         loop: "{{ lookup('subelements', my_hosts, 'hosts') }}"

with the group_vars below give the same results

$ cat group_vars/POC_ENV.yml

Yehuda_Pinhas · December 30, 2019, 12:23pm

Alright so this is how my playbook looks like now:

(attachments)

vbotka · December 30, 2019, 1:18pm

Wrong indentation. Fix:

     my_hosts:
       - group: 'POC_ENV'
         hosts:

HTH,

-vlado

Yehuda_Pinhas · December 30, 2019, 1:39pm

Alright!
Thanks for your help so far, now it’s working without errors.
Ignore previous messages and lets talk about the current status.

The result I get is:

The result I’m expecting to see is that the ansible will run on each line inside POC_ENV.yml the task show_vlan.
The task show_vlan redirects to the “roles” folder where there is a show_vlan.yml file with the commands I want to execute on the remote host (from POC_ENV per line)
I want to store the result of the show_vlan.yml in a variable and print it.

This is the show_vlan.yml content:

name: Show VLAN
ios_command:
commands: show vlan brief
register: show_vlan
debug: var=show_vlan.stdout_lines

What am I missing next?

vbotka · December 30, 2019, 1:52pm

The output of the task "add_host" is weird. "item.1" is empty in couple
of lines or keeps the variables instead of the host's name.

(I hesitate to hope that it helps)

-vlado

j.r.hawkesworth · December 31, 2019, 9:15am

I suggest using a directory that contains the files you need to build up your inventory. You can also use symlinks I think

This is described in the documentation in the section starting 'Aggregating inventory sources with a directory’ here:

https://docs.ansible.com/ansible/latest/user_guide/intro_inventory.html#using-multiple-inventory-sources

I think this might be a simpler way to achieve what you want rather than using add_host and including lots of inventory files directly into your playbook.

It does depend on not having hostnames that clash of course.

Hope this helps,

Jon

Yehuda_Pinhas · December 31, 2019, 10:13am

Hi J Hawkesworth,
Can you explain exactly in which file do I need to add what code?

Considering I have the inventories POC_ENV.yml, avaya.yml, nexus.yml for example. What file do I need to edit? the ansible.cfg and the inventory file? can you provide the exact solution?

currently my ansible.cfg file directs to /etc/ansible/inventory.yml

and my inventory.yml is empty (which is the master inventory as i understand and from it I am supposed to direct ansible to each one of the inventories mentioned above such as POC_ENV.yml,etc.)

J Hawkesworth:

j.r.hawkesworth · December 31, 2019, 10:24am

I suggest doing this:

mkdir /etc/ansible/inventory

copy POC_ENV.yml, avaya.yml, nexus.yml to /etc/ansible/inventory

changing your ansible.cfg so that it looks for inventory in

/etc/ansible/inventory

IIRC that will load all the inventory files in that directory

I have not tried this using .yaml format inventory file but it works fine using ini format so I suspect it will work ok using .yaml format inventory.

If you run
ansible-playbook -vvvvvv a_test_playbook.yml

the first few lines will show you where it is looking for inventory and what plugins it is using to parse and load your inventory.

Hope this helps,

Jon

Yehuda_Pinhas · December 31, 2019, 2:22pm

Hi,
I have followed the instruction and it’s working so my inventories are now working like I wanted. My end goal is to ssh to the specified machine in POC_ENV inventory (TEST-AGG-SW) and run the command “show vlan brief” and return it to stdout.
For some reason it is not working and im not even sure if my code runs the show vlan br but not post it to stdout or not sure if it even performs the SSH to the switch or not.

My configuration:

This is my playbook:

(attachments)

j.r.hawkesworth · December 31, 2019, 9:03pm

Hi

I can see a couple of things that might need changing to get this to work:

Try changing your playbook so it looks like this:

`

Yehuda_Pinhas · January 2, 2020, 8:07am

Hi Jon,
Thank you so much, it’s working now.

Also, much thanks Vladimir for his assistance.

Take care.

Topic		Replies	Views
In v2.4.x.x how can I include host groups in one inventory file to another? If so how? Ansible Project	4	0	December 6, 2017
Hello guys, I'm unable to pass a dynamic inventory script to read a yml file from repo Ansible Project	4	3	September 26, 2019
add_host in ansible Ansible Project	1	2	October 5, 2017
best practices re ansible-playbook inventories and group_vars Ansible Project aws	1	5	August 20, 2019
build groups based on hosts and host-variables Ansible Project windows	5	3	July 2, 2024

Need help on building ansible inventory file properly

copy POC_ENV.yml, avaya.yml, nexus.yml to /etc/ansible/inventory

Related topics