How to load external group\host_vars into playbook.

Get Help
1

Hello community,
Need your advice. Before ansible 2.16 we were able to load host_vars and group_vars so to say “dynamically” on the fly from external repository. The idea here was to store playbooks and variables separately from each other, due to some requirements.

Here is an example:

- hosts: alerta
  pre_tasks:
  - name: Fetch vars
    local_action:
      module: git
      repo: https://{{ gitlab_ansible_user | urlencode }}:{{ gitlab_ansible_pass }}@gitlab.example.com/ansible/vars/alerta.git
      dest: /etc/ansible/gitlabvars/alerta
    run_once: true
    no_log: true
    changed_when: false
  - name: Copy group_vars to playbook root
    local_action:
      module: copy
      src: /etc/ansible/gitlabvars/alerta/group_vars
      dest: ./
  roles:
    - alerta

The structure of alerta vars repository (gitlab.example.com/ansible/vars/alerta.git) is following

/vars/alerta/group_vars$ tree 
.
├── alerta_dev.yml
├── alerta_prod.yml
├── alerta_rc.yml
└── alerta.yml

Unfortunately since 2.16 that type of setup is no longer valid. According to release notes, host_vars and group_vars currently are loaded only single time when playbook is started, so host_vars and group_vars must be already be in place when playbook is launched.

Have anyone faced something similar to this? Are there any traditional ansible way of loading these variables in to the playbook ?
p.s include_vars has some drawbacks that will require you to create logic related to precedence of variables.

Thanks in advance!

2

I would suggest making your pre-tasks a separate playbook to get the group files in place, then as a separate play in that playbook, use import_playbook to invoke the playbook that calls your alerta role.
See ansible-doc -t module ansible.builtin.import_playbook, and do let us know how that works for you.

Good luck!

3

Hey!
Thanks! It seemed at the first glance that it should do the trick indeed, but unfortunately no luck for now.

ansible [core 2.19.3]

Created simple test playbook:
ansible-playbook prepare_env.yaml --inventory ../hosts

#prepare_env.yaml
- name: Prepare env
  hosts: localhost
  gather_facts: true
  tasks:
  - name: copy vars
    ansible.builtin.copy:
       src: vars/
       dest: ./
- ansible.builtin.import_playbook: set_facts_debug.yaml

and

---
#set_facts_debug.yaml
- name: Example playbook to set facts and show debug messages
  hosts: localhost
  gather_facts: true
  tasks:
    - name: "Debug: show application name"
      debug:
        msg: "Application name is: {{ app_name }}"

    - name: "Debug: show full facts in structured output"
      debug:
        var: app_version

/vars$ tree
.
└── group_vars
    ├── demo
    │   └── demo.yaml
    ├── demo_abc
    │   └── demo_abc.yaml
    ├── demo_dv
    │   └── demo_dv.yaml

Imported playbook is unaware of variables that were copied in the parent playbook, so it fails.
p.s. on the second run, when vars are already in place, all works good.

So I am just thinking now is our setup anti-pattern or that’s a bug in ansible ?

4

Perhaps a little bit of both. (?) It obviously isn’t a common way to do things.

Now I’m wondering about the nature of the data. The mechanism that keeps host and group data separate from the Ansible projects that uses them is host facts. Could you add enough Ansible to the data repos to push them out to hosts as facts? Then your “real” playbooks could pull them in with normal fact gathering. That would allow the data project’s modification patterns to be quite different from that of the code project(s).

To be honest, though, that difference of pacing is all I’ve been able to imagine as an advantage for this scheme. I only have our own projects to go by, but my feeling is that the structure of the data and the code that uses them is so thoroughly intertwingled that separating them into separate projects seems borderline questionable.

One more thought that may allow you to keep using them as you have is to use git submodules — put your data project inside your code project at the proper place, but keep them as separate repos. I’ve never done it, but it seems like it might work.

5

According to some comments I’ve seen since writing the above, apparently there are “issues” (?) getting git submodules working reliably under AWX / AAP. If you intend to run this on AWX, that should be tested thoroughly first.