Host_vars adjacent to playbook_dir for playbooks from a collection

Project Discussions
, ,
1

I’m moving a bunch of playbooks to a collection, so I can share them across multiple projects.

And I’m running into something I did not expect:

I use vaults in host_vars and group_vars directories in the project directory, specifically to let Ansible Automation Platform (awx) handle them (for some reason, Red Hat thinks it should not be allowed to use encrypted vault files in your inventory, although you can use encryptString). Anyway, this works fine for me so far.

├── playbook.yml
├── collections
├── inventory
│   ├── inventory1
│   │   ├── group_vars
│   │   │   └── DEV
│   │   └── host_vars
│   │       └── server1
│   └── inventory2
── group_vars
│   ├── all
│   └── DEV
├── host_vars
│   └── server1
├── templates
├── roles
└── files

Unfortunately, this does not work for playbooks I run from the collections (I use FQDN to run the playbook).
So in this example, the playbook.yml would contain an import_playbook line.

- name: import playbook
  import_playbook: my.namespace.playbook1

The reason for the playbook importing a playbook from the collection , is also in AAP, where it’s not (yet) possible to start playbooks from a collection.

The reason is kinda obvious, the playbook_dir in the playbook in the collection, is not the playbook_dir for my project, but a directory in the collection. So it cannot locate my host_vars/group_vars directories with the encrypted vault files.

I would like to see an option where I can select what the playbook directory (as used by the host_group_vars plugin) needs to be when using playbooks from a collection (eg. something like PLAYBOOK_DIR_FROM_COLLECTION = false)

3 Likes
2

I’ve got a workaround now for this:

  • before running collection playbooks, I copy the host_vars and group_vars folders into the collection’s playbooks directory (using ansible copy).
  • the copied vaults are no longer encrypted in the copied location
  • debug shows that the secrets from the vaults are now visible
  • for some reason, I have to use set_fact to actually make the secrets available in roles (?)

I still have to verify this on awx/aap, though …

This does not work on AAP, though, atm.

Update

I’m now creating a symlink in the collection’s playbooks directory, and this works on both the Ansible CLI and in AAP.
So I’m symlinking {{ playbook_dir }}/group_vars using the file module to /runner/requirements_collections/ansible_collections/mynamespace/mycollection/playbooks/group_vars …
This works, but it’s pretty ugly.

2 Likes
3

Hello Tom,

I am trying to do somewhat the same with packaging the same playbooks in a collection to avoid duplication.

And found the following:

Group/Host Vars at the inventory level will always be read, if you put your variables at the inventory level you’re safe.

But Group/Host Vars at Play will always be ignored, and even considered undefined when forcing the variable value at the import level.

Which is probably why the symlink you’re doing is working. As the imported/collection playbook will look at it’s own local folders.

I am sure there is a way to tweak the host_group_vars plugin to force it to look at the playbook directory to include the vars required.

And do agree that there should be a variable to force where variable are looked up for a play, local, collection, or both.

For me i am not sure i will do the symlink stuff seems lack hackish solution that could break at any moment ^^

4

Before it would be possible to add a toggle to the host_group_vars plugin, we’d need to track the playbook hierarchy, like this attempt Add Playbook and PlaybookInclude into parent chain by sivel · Pull Request #73309 · ansible/ansible · GitHub.

I don’t think there’s a risk to Ansible breaking the symlinks use case (but agree it’s not great).