galaxy.ansible.com cloudflare down awx broken

Get Help
, , , ,
1

currently with galaxy.ansible.com down due to a Cloudflare issue in Dallas my awx project is broken as it cannot pull in anything for the requirements.yaml file.

collections:

  • name: community.general
  • name: awx.awx

I get a 500 error (site is down) when syncing my git repo for my playbooks as it cannot update the community.general and awx.awx collections

I had to rush to disable all plays and tasks related to these so that the project could sync but is there a way to specify some sort of mirror or be able to pull these from the git repositories instead of galaxy.ansible.com?

This could have affected a large number of systems today and by the time i found this i only had about 30 to 45 minutes to disable the plays and tasks related to these collections.

Any alternatives would be greatly appreciated.

3 Likes
2

Do you have another copy of these collections somewhere, maybe checked out on a laptop?

If you can’t wait for Cloudflare to recover, you could try explicitly checking the third-party roles into git, bypassing pulling them dynamically.

You could grab the roles from their Github repos as another option.

It may be unconventional, but checking in third-party roles is my preferred way to manage the few I need, partly because it solves the problem of “what if the upstream copy becomes inaccessible?”

3

That’s a good idea but i haven’t figured out how to get the roles to be recognized in awx yet. I have created some roles in my awx playbook project but it doesn’t see them under the roles directory.

from what i have found the directory tree is this:

/var/lib/awx/projects/my_project/
├── ansible.cfg
├── inventories/
│ ├── production/
│ │ ├── hosts
│ │ ├── group_vars/
│ │ └── host_vars/
│ └── staging/
│ ├── hosts
│ ├── group_vars/
│ └── host_vars/
├── playbooks/
│ ├── site.yml
│ └── webserver_setup.yml
├── roles/ < nothing underneath is callable by files under playbooks/
│ ├── common/
│ │ ├── tasks/
│ │ └── defaults/
│ └── webserver/
│ ├── tasks/
│ └── templates/
└── README.md

4

I haven’t tested this, but you might need to (temporarily) remove those roles from your requirements.yml so nothing is trying to pull them from remotes.

5

Try updating your ansible.cfg to explicitly add the roles_path you want.

[defaults]
# Keep any paths that there and append this path after a colon
roles_path = /var/lib/awx/projects/my_project/

You might also try the path ../roles to reflect that the roles dir is a sibling to your playbooks dir. See also the docs on search paths:

6

… and now https://galaxy.ansible.com/ui/ is back up. :smiley:

7

Unfortunately it still doesn’t see it:

ERROR! the role ‘debug_role’ was not found in /runner/project/playbooks/Tests/roles:/runner/requirements_roles:/var/lib/awx/projects/_8__tester_project/roles:/runner/.ansible/roles:/usr/share/ansible/roles:/etc/ansible/roles:/runner/project/playbooks/Tests
The error appears to be in ‘/runner/project/playbooks/Tests/test-role.yaml’: line 10, column 7, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
roles:
- debug_role # noqa: syntax-check[specific]
^ here

it’s in the /var/lib/awx/projects/_8__tester_project/roles directory

bash-5.1$ ls -ahl
total 4.0K
drwxr-xr-x. 4 awx root 44 Nov 18 15:53 .
drwxr-xr-x. 7 awx root 4.0K Nov 18 16:21 ..
drwxr-xr-x. 8 awx root 105 Nov 18 15:53 debug_role

bash-5.1$ cd debug_role/
bash-5.1$ ls -ahl
total 4.0K
drwxr-xr-x. 8 awx root 105 Nov 18 15:53 .
drwxr-xr-x. 4 awx root 44 Nov 18 15:53 ..
drwxr-xr-x. 2 awx root 22 Nov 18 15:53 defaults
drwxr-xr-x. 2 awx root 22 Nov 18 15:53 handlers
drwxr-xr-x. 2 awx root 22 Nov 18 15:53 meta
-rw-r–r–. 1 awx root 1.3K Nov 18 15:53 README.md
drwxr-xr-x. 2 awx root 22 Nov 18 15:53 tasks
drwxr-xr-x. 2 awx root 39 Nov 18 15:53 tests
drwxr-xr-x. 2 awx root 22 Nov 18 15:53 vars
bash-5.1$

bash-5.1$ cd tasks
bash-5.1$ cat main.yml

tasks file for debug_role

  • name: Debug passed in variable
    ansible.builtin.debug:
    msg: “{{ debug_me }}”
    notify: It Worked
    bash-5.1$
8

When i run it on a host with ansible installed (host meaning it has the git repo on it and the roles_path in the global ansible config and this is outside awx) it runs fine but inside awx it can’t find the role

9

(post deleted by author)

10

Ok got it to work with the following ansible.cfg in the root of the git project:

[defaults]
# Keep any paths that there and append this path after a colon
roles_path = /runner/project/roles