I did occur this issue before when I run ansible-playbook command to run the playbook before, when this issue occurred, I installed the collection for the user who runs the command, and the issue could be fixed.
When I used AAP and this issue occurred again, I suspected that the user account AAP was using to run playbook might have NO access to the collection, so I tried to run the ansible-playbook command with the accounts awx and the user account who triggered the job, they all could run the playbook successfully.
At this point, I suspect AAP running playbook with a user account unknow, and the user has NO access to the collection, so the issue occurred.
Appreciate if anyone can help on this? Thanks in advance!
My AWX Job Templates are playbooks stored in a project that is hosted on GitLab. My solution was to install the Galaxy Collection to the project directory and sync it back to GitLab.
To make it work:
Create an ansible.cfg file in the same directory as your playbook.
Add this line to the file: collections_path = collections
Create a directory named collections in the same directory as your playbook
Run the ansible-galaxy collection install <YOUR_COLLECTION> to install the collection to your new collections folder.
Sync your project to your git/scm. This will push the installed collection to your repo.
Go into AWX/AAP/Tower and re-sync your project to pull the new collection installation to your AWX/AAP/Tower installation
Run your playbook again and see if that work.
There are other ways you are supposedly able to tell AWX/AAP/Tower to download collections as-needed or with a requirements.yaml file, but I was never able to get that to work.
In your GitHub/GitLab/SCM repository, add a collections/requirements.yml or roles/requirements.yml file with the additional content that you need. AWX/AAP will automatically download it from the galaxy server configured for your org, unless your requirements.yml file specifies a particular source.
Sorry I did not make it clear enough, I did NOT use GIT, just added playbook on my AAP project directly, and then I created a job to run a playbook in that directory
What’s interesting is that I’ve already installed the needed collection (my collection needs to be installed for different account respectively, so I installed for the accounts like awx and my account) on my AAP server, if I run as these users and run ansible-playbook command directly on my AAP server, I could see it runs as expected.
But if I launch the job (it runs the playbook) from AAP console, it gave me the error about “the role was NOT found”.
It looks weird why it worked differently when I run the book by command directly or from AAP console, per the error message on the picture, it looks like when AAP console runs the playbook, it failed to find the role in these directories, if I run command directly, it could find.
Thanks for you reply, in fact, I did NOT use use SCM, in fact, I installed collection on my AAP server manually, and it weird, if I run the ansible-playbook command directly on AAP server, it gave me the expected result, if I run it from AAP CONSOLE (by lauching a job that trigger the playbook), it gave me the error in my picture.
I’m not sure how to solve the underlying problem, but a quick way to get it up and running would be to put the ansible-galaxy command in a playbook and then run the playbook from AAP Console. It would force the installation of the package to the collection folder used by the AAP Console environment.
I don’t have my AAP environment yet. So, I don’t know much about AAP specifically yet.
Are you using AAP on Kubernetes/Docker or are you using the RPM based single node install?
Typically, AAP/AWX run jobs on an execution image (AWX-EE for e.g.), which only contain a specific set of collections. Extra collections can be installed dynamically at runtime (using the method I described), or by having the collections installed in your project folder (like @Dustin described), or by building custom EE images and installing it there.
Installing collections “manually” on the platform itself isn’t straightforward, and probably doesn’t work like you think it does on k8s/docker. If you’re using the RPM single-node install, that might work ok, but the containerized versions would need you to install collections to a shared volume that every EE container would mount.
Thanks for your reply.
I fixed the issue by copying the missing collections to the directory /var/lib/awx/projects/[my project name]/collections, it seems that when AAP runs the job, it only read the collections from that directory.
My issue with requirements.yml was that I couldn’t figure out how to get EE to trust or ignore my company’s proxy cert. So, I just install my packages manually into the projects.
Ah. That can be fixed at least a couple different ways. One is to build your own EE and install the root CA onto it directly, which is what I do.
The other option is to make sure the root CA is added to the AWX custom resource, and telling the EE environments in the UI use the CA bundle. The control plane should pick up the CA automatically, but I’ve found that the execution pods don’t seem to do the same. I think you just have to add the following environment variable to fix it.
I’ll give this a try when I get back in office on Monday. My biggest hurdle with this was that I was learning Kubernetes, docker, pods, ansible, and awx all from scratch at the same time to get AWX-Operator working.
Now that I have a working AWX, I’m going back through my dev environment and making my deployment more efficient/optimized. This is one of the areas I definitely want to tackle to achieve that. I don’t like having to manage my collections in every venv I build…
Next after this is figuring Ansible Development Server with ansible-navigator and VSCode server.
