# AWX Not Downloading Role Requirements **URL:** \ **Category:** AWX Project\ **Tags:** awx\ **Created:** [September 9, 2019, 12:58pm UTC](https://forum.ansible.com/t/awx-not-downloading-role-requirements/31235 "2019-09-09T12:58:32Z")\ **Posts on this page:** 14\ **Page:** 1
**Author:** ![Fran\_Fitzpatrick](https://avatars.discourse-cdn.com/v4/letter/f/7ab992/32.png) [@Fran\_Fitzpatrick](https://forum.ansible.com/u/Fran_Fitzpatrick)\ **Post date:** [September 9, 2019, 12:58pm UTC](https://forum.ansible.com/t/awx-not-downloading-role-requirements/31235/1 "2019-09-09T12:58:32Z")
Hi all, I’m noticing this on v7.0.0 (didn’t see it on 6.0.0; hadn’t tried 6.1.0) where my role requirements aren’t being downloaded. Has anyone else seen this? Detailed issue is described here: [https://github.com/ansible/awx/issues/4682](https://github.com/ansible/awx/issues/4682) Fran ---
**Author:** ![AlanCoding](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/alancoding/32/301_2.png) [@AlanCoding](https://forum.ansible.com/u/AlanCoding)\ **Post date:** [September 9, 2019, 1:22pm UTC](https://forum.ansible.com/t/awx-not-downloading-role-requirements/31235/2 "2019-09-09T13:22:40Z")
This is an expected change of behavior as a result of a recent feature merge. The roles are downloaded before a job runs, not in the project update. Have you experienced any issues running jobs that use those roles? ---
**Author:** ![Fran\_Fitzpatrick](https://avatars.discourse-cdn.com/v4/letter/f/7ab992/32.png) [@Fran\_Fitzpatrick](https://forum.ansible.com/u/Fran_Fitzpatrick)\ **Post date:** [September 9, 2019, 1:59pm UTC](https://forum.ansible.com/t/awx-not-downloading-role-requirements/31235/3 "2019-09-09T13:59:45Z")
Eep. I realized I hadn't been hitting Reply All, so I'm CC'ing the mailing list with another response. Anyway, to compare that output to another AWX instance of mine with v6.0.0: === ======= ======================== ========== ======== =========================== ======= id project name status job\_type created elapsed === ======= ======================== ========== ======== =========================== ======= 146 88 redacted-master successful run 2019-09-04T14:07:14.679296Z 7.714 148 88 redacted-master successful check 2019-09-04T14:08:32.529242Z 13.889 149 88 redacted-master successful run 2019-09-04T14:08:54.488568Z 7.88 151 88 redacted-master successful check 2019-09-04T14:22:47.037406Z 14.505 152 88 redacted-master successful run 2019-09-04T14:23:09.578303Z 6.978 154 88 redacted-master successful check 2019-09-04T14:27:31.961189Z 14.0 155 88 redacted-master successful run 2019-09-04T14:27:53.945092Z 7.277 157 88 redacted-master successful check 2019-09-04T14:45:55.339746Z 23.42 158 88 redacted-master successful run 2019-09-04T14:46:26.798998Z 7.286 160 88 redacted-master successful check 2019-09-04T14:49:58.269942Z 21.962 161 88 redacted-master successful run 2019-09-04T14:50:28.924425Z 6.979 163 88 redacted-master successful check 2019-09-04T14:51:39.157141Z 14.642 164 88 redacted-master successful run 2019-09-04T14:52:02.002206Z 7.077 166 88 redacted-master successful check 2019-09-04T14:53:49.383286Z 13.834 167 88 redacted-master successful run 2019-09-04T14:54:11.412924Z 6.832 169 88 redacted-master successful check 2019-09-04T14:56:52.642154Z 13.732 170 88 redacted-master successful run 2019-09-04T14:57:14.373014Z 7.159 172 88 redacted-master successful check 2019-09-04T14:59:00.789157Z 15.496 173 88 redacted-master successful run 2019-09-04T14:59:24.397416Z 7.107 175 88 redacted-master successful check 2019-09-04T15:10:48.067277Z 63.171 176 88 redacted-master successful run 2019-09-04T15:11:59.069144Z 8.684 178 88 redacted-master successful check 2019-09-04T15:22:39.657026Z 14.556 === ======= ======================== ========== ======== =========================== ======= Here you can see I'm getting both `check` and `run` project\_update jobs. ---
**Author:** ![AlanCoding](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/alancoding/32/301_2.png) [@AlanCoding](https://forum.ansible.com/u/AlanCoding)\ **Post date:** [September 9, 2019, 3:14pm UTC](https://forum.ansible.com/t/awx-not-downloading-role-requirements/31235/4 "2019-09-09T15:14:43Z")
Could you do a quick debugging test for me? On your project, if you uncheck the box to allow override of branch, then does it download requirements? Also, when you encountered this bug, what was your branch value, both on the project & the job template? I may be close to putting together a picture. ---
**Author:** ![Fran\_Fitzpatrick](https://avatars.discourse-cdn.com/v4/letter/f/7ab992/32.png) [@Fran\_Fitzpatrick](https://forum.ansible.com/u/Fran_Fitzpatrick)\ **Post date:** [September 9, 2019, 4:27pm UTC](https://forum.ansible.com/t/awx-not-downloading-role-requirements/31235/5 "2019-09-09T16:27:44Z")
I don't think I've ever been as bad at hitting 'Reply All' as I am with doing it in this thread. Adding the mailing list back in. Shame on me. ---
**Author:** ![AlanCoding](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/alancoding/32/301_2.png) [@AlanCoding](https://forum.ansible.com/u/AlanCoding)\ **Post date:** [September 9, 2019, 4:31pm UTC](https://forum.ansible.com/t/awx-not-downloading-role-requirements/31235/6 "2019-09-09T16:31:19Z")
If you go to the job in the API (`/api/v2/jobs/42/`), look into the “related” dictionary, is a link present under the key “project\_update”? If that is not present and your project has the `roles/requirements.yml` file, then that’s a bug. If you are on the same page as me about these facts, then I’m happy to re-open your issue. My challenge for pinning down this issue is that I can’t yet create the situation you saw. This is a reference case in a public repo: [https://github.com/ansible/test-playbooks/tree/with\_requirements](https://github.com/ansible/test-playbooks/tree/with_requirements) I’m happy to re-run this and give you any details from my server to compare, to help find where they diverge. The most valuable information for me me would be if you could turn the log level to DEBUG (you can do this in the UI settings under logging), run your job, and then get the server logs. I very specifically expect to see a message like “Running project sync for job 67 because of galaxy role requirements”. Is the project type git? Thanks, ---
**Author:** ![Fran\_Fitzpatrick](https://avatars.discourse-cdn.com/v4/letter/f/7ab992/32.png) [@Fran\_Fitzpatrick](https://forum.ansible.com/u/Fran_Fitzpatrick)\ **Post date:** [September 9, 2019, 4:54pm UTC](https://forum.ansible.com/t/awx-not-downloading-role-requirements/31235/7 "2019-09-09T16:54:40Z")
> Is the project type git? Yup. 🙂 So I launched a new job from a template. It spawned Job #61. This triggered Project Update #62. Job 61s related dictionary: "related": {         "created\_by": "/api/v2/users/1/",         "labels": "/api/v2/jobs/61/labels/",         "inventory": "/api/v2/inventories/4/",         "project": "/api/v2/projects/8/",         "extra\_credentials": "/api/v2/jobs/61/extra\_credentials/",         "credentials": "/api/v2/jobs/61/credentials/",         "unified\_job\_template": "/api/v2/job\_templates/9/",         "stdout": "/api/v2/jobs/61/stdout/",         "job\_events": "/api/v2/jobs/61/job\_events/",         "job\_host\_summaries": "/api/v2/jobs/61/job\_host\_summaries/",         "activity\_stream": "/api/v2/jobs/61/activity\_stream/",         "notifications": "/api/v2/jobs/61/notifications/",         "create\_schedule": "/api/v2/jobs/61/create\_schedule/",         "job\_template": "/api/v2/job\_templates/9/",         "cancel": "/api/v2/jobs/61/cancel/",         "relaunch": "/api/v2/jobs/61/relaunch/"     } I just realized something... what path does AWX detect `roles/requirements.yml` from? It it "{path of playbook}/roles/requirements.yml" (probably unless otherwise configured differently from ansible.cfg)? I ask because with this project that I'm working with, the playbooks aren't at the root of the directory. It would be something like "{git project root}/ansible/roles/requirements.yml" and the playbook is "{git project root}/ansible/my\_awesome\_playbook.yml". As for your other logs, I'll start working on collecting them for ya. ---
**Author:** ![Fran\_Fitzpatrick](https://avatars.discourse-cdn.com/v4/letter/f/7ab992/32.png) [@Fran\_Fitzpatrick](https://forum.ansible.com/u/Fran_Fitzpatrick)\ **Post date:** [September 9, 2019, 5:58pm UTC](https://forum.ansible.com/t/awx-not-downloading-role-requirements/31235/8 "2019-09-09T17:58:03Z")
Alan, See the debug logs here at this gist (retrieved by docker-compose logs -f task): [https://gist.github.com/fxfitz/0640924cfd5c3f9540e4fa6ab76d8f70](https://gist.github.com/fxfitz/0640924cfd5c3f9540e4fa6ab76d8f70) The template job is #63, and the project\_update is #64. As for anything related to a project sync, this is what I see: awx.main.tasks Skipping project sync for job 63 (running) because commit is locally available Fran ---
**Author:** ![AlanCoding](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/alancoding/32/301_2.png) [@AlanCoding](https://forum.ansible.com/u/AlanCoding)\ **Post date:** [September 9, 2019, 6:31pm UTC](https://forum.ansible.com/t/awx-not-downloading-role-requirements/31235/9 "2019-09-09T18:31:50Z")
> I just realized something… what path does AWX detect `roles/requirements.yml` from? It it “{path of playbook}/roles/requirements.yml” (probably unless otherwise configured differently from ansible.cfg)? AWX will only consider `roles/requirements.yml` relative to the root of the project. There is no option to configure this. If you still have that server running with an older version, it might help to look at what this was actually doing before the upgrade. In your logs, project update 64 was launched as a “dependency”, and this is not what I was looking for. Your logs confirm that the project update needed to install rolls did not run. ---
**Author:** ![Fran\_Fitzpatrick](https://avatars.discourse-cdn.com/v4/letter/f/7ab992/32.png) [@Fran\_Fitzpatrick](https://forum.ansible.com/u/Fran_Fitzpatrick)\ **Post date:** [September 9, 2019, 6:48pm UTC](https://forum.ansible.com/t/awx-not-downloading-role-requirements/31235/10 "2019-09-09T18:48:13Z")
So, what's the next step forward? Reopening the issue? FWIW, I just added another project that does have "{git repo root}/roles/requirements.yml" and also observed that it did not download roles; once again `roles_enabled=False` in `project_update.yml`. ---
**Author:** ![AlanCoding](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/alancoding/32/301_2.png) [@AlanCoding](https://forum.ansible.com/u/AlanCoding)\ **Post date:** [September 9, 2019, 7:34pm UTC](https://forum.ansible.com/t/awx-not-downloading-role-requirements/31235/11 "2019-09-09T19:34:24Z")
You previously mentioned the project parameters: > “Clean”, "Update Revision on Launch (Cache Timeout=0), and “Allow Branch Override” > (and project branch left blank) I will get back to this and test this situation for a project that has “{git repo root}/roles/requirements.yml”. You mentioned that you are still seeing the issue. Do these parameters still apply? ---
**Author:** ![Fran\_Fitzpatrick](https://avatars.discourse-cdn.com/v4/letter/f/7ab992/32.png) [@Fran\_Fitzpatrick](https://forum.ansible.com/u/Fran_Fitzpatrick)\ **Post date:** [September 9, 2019, 8:11pm UTC](https://forum.ansible.com/t/awx-not-downloading-role-requirements/31235/12 "2019-09-09T20:11:41Z")
Yup! They sure do. Thanks for all your assistance! ---
**Author:** ![Fran\_Fitzpatrick](https://avatars.discourse-cdn.com/v4/letter/f/7ab992/32.png) [@Fran\_Fitzpatrick](https://forum.ansible.com/u/Fran_Fitzpatrick)\ **Post date:** [September 10, 2019, 1:42pm UTC](https://forum.ansible.com/t/awx-not-downloading-role-requirements/31235/13 "2019-09-10T13:42:39Z")
A few questions and comments so far while I'm also debugging locally: 1) How come the change from using `project_update.yml` to the RunJob's `pre_run_hook`? I'm just curious as to the reasoning to the move simply because the other way had much easier visibility into the problem, not to mention having both makes it even more confusing (meaning having the pre\_run\_hook and tasks in project\_update.yml doing the same thing). For me personally, I liked being able to see the roles being updated in `project_update.yml`. 2) It's becoming more and more apparent to me that AWX should not hardcode `roles/requirements.yml`. I see in [https://github.com/ansible/awx/issues/4540](https://github.com/ansible/awx/issues/4540) it's being discussed, but I don't think we should simply add `roles/requirements.yaml` to the search list. I think instead we need to have a user-defined roles/collections requirements path in the project configuration (and perhaps a user-configurable default search list in the main AWX settings). A good example of why #1 is important is the AWX project itself. For example, if the AWX installer were ever to have an external role requirement, the AWX application couldn't run the AWX installer. Currently, AWX would look for `roles/requirements.yml` and wouldn't find it, but when the install.yml playbook was run it would fail because external roles wouldn't be downloaded. If we could configure the "search paths" on a per-project basis, we could eliminate this. What do you think? ---
**Author:** ![AlanCoding](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/alancoding/32/301_2.png) [@AlanCoding](https://forum.ansible.com/u/AlanCoding)\ **Post date:** [September 19, 2019, 12:42pm UTC](https://forum.ansible.com/t/awx-not-downloading-role-requirements/31235/14 "2019-09-19T12:42:00Z")
I took a fresh stab at this, and this is how it behaves for me: Create a new project with roles/requirements.yml in default branch and set to update on launch (and some other parameters you mentioned) this automatically causes a “check” type project update Make a JT using that project and launch it this automatically causes a “check” type project update because it updates on launch the JT pre\_run\_hook triggers a “run” type project update (this updates roles) I can’t replicate any bug with this. The job template runs some tasks that requires the roles to be installed in order to pass. > For me personally, I liked being able to see the roles being updated in `project_update.yml`. That was not changed. There is a line between what is managed in the `pre_run_hook` tasks and what is managed in the project\_update.yml playbook. Anything that talks to a remote server must be done in the project\_update.yml so that it has visibility to the user (in the project\_update.yml standard out). The pre\_run\_hook python logic should only involve internal tasks (including moving things around on the local file system). If this separation is not maintained, that’s a bug. I don’t disagree with anything in your point (2). You should leave a comment in that github issue with your design input so it doesn’t get lost in this email reply chain.