Possible to use a collection's playbook as a project?

Get Help
,
1

I authored a collection. The collection has two roles (wma.general.a, wma.general.b) and two playbooks (playbook a performs include_role on wma.general.a and playbook b performs include_role on wma.general.b).

Does anyone have any thoughts on how this collection could be used in AWX as a project without having to have a separate git project that’s a wrapper playbook that has a dependency on my wma.general collection and also has playbooks that are wrappers for the collection’s roles?

I’d like to be able to use the playbook within the collection, but it doesn’t seem to be possible. For one, when I sync the collection as a project, AWX does not look at the collection’s dependencies in galaxy.yml so any upstream collections my collection depends on don’t get installed and if I try to use the collection’s playbook, AWX complains that the role the playbook is trying to run cannot be found.

The only workaround for this if this were to be completely self-contained, would be to have the collection depend on itself…which would be terrible :slight_smile:

Does anyone have any thoughts or am I not making any sense here?

2

From what I know about AAP/AWX, I don’t think it’s possible without a playbook, or at least a symlink to a playbook.

I’m guessing you can also ‘just’ symlink the playbook to your project, that’s still some extra work, but at least it will update the playbook when the collection is updated :slight_smile:

3

Where would the symlink be in this case? Where would it live, I mean?

4

Depending on the structure of your project, anywhere where AWX/AAP can reach it. I use the following structure (and you can guess where all the playbooks go):

.
|-- ansible.cfg                         # Ansible config
|-- cache                               # Fact cache
|-- collections                         # Collections required for the project
|-- files -> playbooks/files            # Convenience symlink to files
|-- group_vars -> inventory/group_vars  # Convenience symlink to inventory vars
|-- host_vars -> inventory/host_vars/   # Convenience symlink to inventory vars
|-- inventory                           # Directory with inventory files
|-- library                             # Any extra modules/plugins etc.
|-- playbooks                           # Playbooks
|-- requirements.yml                    # Ansible Galaxy requirements (roles and collections)
|-- roles                               # Roles required for the project
`-- scripts                             # Auxillary scripts for the project

GitHub - Thulium-Drake/ansible-role-ansible_controller for more information

5

I kind of see what you mean, but would this translate to ansible-galaxy building my collection, publishing it to internal automation hub and then AWX/AAP pulling that tar.gz down from AH and still being able to use the symlinks?

Keep in mind that we don’t do anything on the controller (multi-machine AAP controller nodes in a cluster) besides creating the cluster inventory and running the setup script, so everything has to be done externally

6

Yes, as long as the collection_path in ansible.cfg is the same, the same collection with the same name will always have the same path :slight_smile:

So you could have a playbook in myorg.mycoll which would be located at:

<project>/collections/ansible_collections/myorg/mycoll/playbooks/playbook.yml

And it doesn’t matter which source the collection comes from, which version of Ansible you run (* as long as it supports collections) or which project it is. It will however still pick up whatever you have set up in the inventory.

7

I’ve been doing a little more research into this and I wonder if this would also work:

Essentially, in your collection’s playbook you have a collections parameter that is self-referencing to the collection the playbook lives in: netapp.ontap/playbooks/examples/na_ontap_pb_upgrade_firmware_with_vars_file.yml at main · ansible-collections/netapp.ontap · GitHub

Alternatively, I also wonder if it’s possible to have a /collections subdir in your /playbooks dir in your collection with a requirements.yml that also self-references the collection?

re: Using collections in a playbook — Ansible Community Documentation

8

I’m trying to understand what you’ve gained by packaging this pair of roles and playbooks that use them as a collection rather than as a project with a couple of local roles. I can think of only a couple of reasons to make a collection: (1) to package chunks of “Ansible functionality” for use by other projects, and (2) to avoid needless duplication of python code that can be shared easily by plugins within a collection but that would be difficult or impossible to share if not packaged in a collection. (I’m probably overlooking something huge, but #1 is extremely broad, #2 is tediously specific, and at the moment I can’t think of anything that needs a 3rd category.)

How likely are these roles to be used across multiple projects? If it is at all likely, then why not turn the two playbooks into examples in the collection’s README.md, and let the projects which need those roles list the collection in their collections/requirements.yml, and have the playbooks live in such “normal” projects?

I have always been skeptical of the benefits of pre-packaged playbooks, yet I’m open to persuasion.

2 Likes
9

We’re looking into creating collections since that’s what can be easily shared with other internal groups via internal Ansible Automation Hub and very soon, Artifactory. Unfortunately we can’t share roles through these means with other internal groups and the only way to share roles with with other internal groups is mainly through git repositories.

We’re not yet to the point of creating plugins or modules, though that may be a future benefit.

How likely are these roles to be used across multiple projects?

That’s the intent of the collection I’m making, is as a general system maintenance collection which includes narrow purpose roles.

If it is at all likely, then why not turn the two playbooks into examples in the collection’s README.md, and let the projects which need those roles list the collection in their collections/requirements.yml, and have the playbooks live in such “normal” projects?

That’s going to be the path forward. But because it’s our group eating our own dog food, I was hoping to not have to create a separate git project to create a separate playbook if the collection already can have a playbook that can manage the work. It seems like a small thing but it did bite at my curiosity as to whether it could be done or not.

In the end, it looks like the playbooks in my collection that target the roles in the same collection will end up being example playbooks for other projects but also doubling as something that molecule’s converge playbook can use to run the roles within the collection during the testing workflow for the collection.

10

I may be projecting, but I think you’re running into the same problem that has kept lots of our roles from being generally sharable. They work great for us, but would be useless to another shop because they are rife with “group logic”, by which I mean conditionals tailored to our group name and host name conventions. Another shop would have to structure their clusters the same way we have done ours to be able to make use of our very-much-us oriented roles.

To be generally sharable, roles need to be free of “group logic” (or so I’ve convinced myself). That leads me to this guideline, or rule of thumb, or design philosophy – call it what you will – that I have not seen anyone else professing, at least in these terms. It’s intended to help distinguish what should go into a playbook vs. what should go into roles:

  • Roles contain the bits that do stuff.
  • Playbooks determine what assets stuff gets done to, and how.
    You can think of a playbook as a mapping between your inventory and roles.

Part of the art is to be deliberate about violating such rules, which is not possible without awareness of them. But having spelled it out above, I no longer have that excuse. :slight_smile: Trivial tasks of course can reside in a playbook; burying them in a role that would take more work to share than it would to create from scratch gains nothing. But as soon as we move beyond trivial, it pays to head that separation of purposes – at least if there’s ever any intent of sharing the roles.

You initially asked:

…and those are my thoughts. I could ask the same question: Am I making any sense?

11

We maintain the same philosophy with our roles and playbooks as well. We do try to extend as much as possible the same philosophy into playbooks, meaning keeping site-specific content out and instead generalizing them as much as possible to the system or environment (RHEL9, AWS, etc) and allowing end-users or end-systems such as AWX/AAP inject variables for use-case specific runs. This has worked out great for us and groups outside of ours that we share our roles and playbooks with and also translates to very generic testing in Molecule that can (or should) work for groups outside of ours.

This all works within our organization which consists of many groups working in various environments. Outside of our org, our playbooks may still work but may not have a point since they perform a lot of the organizational mandates that probably aren’t a thing outside of the org.

The playbooks also don’t really concern themselves too much with the inventory as that’s mostly handled upstream by AWX/AAP. The playbooks might look at “Am I an EC2 instance. Am I a machine in this data center. Am I Linux? Am I windows?” and branch logic from there. But that may be what you were getting at w/ You can think of a playbook as a mapping between your inventory and roles..

The way we try to structure our playbooks are to be light wrappers around roles. Playbooks, to us, help to organize the order which our roles run and may inject pertinent variables into the roles depending on the environment. We try to not do much in way of tasking in playbooks if that tasking is at all shareable. If it is shareable, we first consider a role it may go into or if a new role can be created around it. If not, then it can go in as a task into the playbook.