I am running AWX inside of some docker containers and can’t get the project_update job to install roles from my requirements.yml file. It successfully finds the file, but, for some reason, scm_full_checkout is always set to false.
I modified /usr/lib/python2.7/site-packages/awx/playbooks/project_update.yml to add this:
So I was able to get this to at least try to install the roles when I run the job. However I am running into a few issues:
The roles aren’t installing with error: “SCM Update failed with ID 37”.
I assume this is because the roles require creds. Unfortunately, the job template doesn’t allow for adding “Source Control” creds.
I tried setting the “machine” creds to use my key, but that failed with “ID 39” (I didn’t actually expect this to work).
The output display is empty (even at highest verbosity).
I assume this is because the output display is only going to show the output from the playbook that I specified for the job.
Is there a way to assign creds to the job template to use when grabbing new roles from a requirements.yml? In this case, it can use the same creds that it used to get the project (same git repo).
Note that there are two different modes for running project updates (job_type == “check” vs. job_type == “run”, which also show up as launch_type == “dependency” vs. launch_type == “sync”). The check/dependency job runs project_update.yml in check mode and sets scm_full_checkout = false, whereas the run/sync mode sets scm_full_checkout = true.
The default jobs list hides the sync updates: /#/jobs?job_search=page_size:20;order_by:-finished;not__launch_type:sync
If you modify that default filter in the URL, you should see both SCM updates that ran before your playbook run: /#/jobs?job_search=page_size:20;order_by:-finished
What Matt meant by “It’ll only sync the roles when a job is run using that project” is that the the requirements.yml will only be pulled down by the second run/sync SCM update prior to running your job template. The SCM credentials specified for your project (if SSH keys) should be available for use by galaxy when checking out roles, but if you’re using username/password instead, we won’t be able to inject that into the SCM URLs as we do for normal SCM updates (https://github.com/ansible/awx/blob/devel/awx/main/tasks.py#L1346).
That helped me find the “SCM Update” task that had the actual error in it. Turns out, the urls in the requirements file were ssh host aliases and (obviously) those aliases aren’t configured in the container.
I also have to send a username (because awx in the container runs as root and user:root isn’t a valid git user).
I specify this username (no password, just the key) in the project definition.
Task: “update project using git and accept hostkey” works fine (meaning it sends ssh://username@git.url), presumably because it builds the URL.
Task: “fetch galaxy roles from requirements.yml” just runs galaxy as -r requirements, so uses the URL from the file, which doesn’t contain a username.
hence, failure.
I don’t see a good AWX or galaxy way around this, though. And, if I put the username in the URL in the requirements file, then nobody else can use that file to install the requirements on their own machine (for dev/testing/etc) without modifying the file to either remove the username or use their own.
I also tested this when in check mode and it also tried (and failed) to install the requirements.
(fwiw, updating the requirements when in check mode is good. Otherwise, every playbook in the project that “requires” a role in the “requirements” file would fail)
anyway, I got everything installed, but then the playbooks can’t find the roles!
My tree:
[root@awx projects]# tree -L 4
.
└── _6__my_projectmikeb_test
├── ansible.cfg
├── inventory_scripts
│ └── cmdb_inventory.py
├── playbooks
│ └── test
│ ├── helloworld.yml
│ └── network_config.yml
├── requirements.txt
└── roles
├── my.libs
├── my.tasks
└── requirements.yml
So … I had to modify my project’s ansible.cfg to set roles_path=roles/ (relative path).
Which leaves me with 2 modifications to my project’s repo to work within AWX, but not anywhere else.
I may just stick with mounting a volume into awx_task:/etc/ansible/roles/ and installing the roles on the host. I’ll have to sleep on it.