"ansible-galaxy role import" is failing with an error

githubixx · October 17, 2023, 9:06pm

Hello,

I’m trying to import a new version of my role with this command (also tried with a Github action, same result - and also created a new token because I got a “403 - permission denied” with the first attempt):

“”"
ansible-galaxy role import --api-key <…> githubixx ansible-role-etcd

Successfully submitted import request 2052239628262167152290618988517025690
running
githubixx.etcd 9.1.0+3.4.7 has already been imported
File “/venv/lib64/python3.11/site-packages/pulpcore/tasking/tasks.py”, line 66, in _execute_task
result = func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File “/app/galaxy_ng/app/api/v1/tasks.py”, line 145, in legacy_role_import
raise Exception(msg)
“”"

“”"
ansible-galaxy --version
ansible-galaxy [core 2.15.4]
config file = None
configured module search path = [‘/usr/share/ansible/plugins/modules’]
ansible python module location = /usr/lib/python3.11/site-packages/ansible
ansible collection location = /usr/share/ansible/collections
executable location = /usr/bin/ansible-galaxy
python version = 3.11.5 (main, Sep 2 2023, 14:16:33) [GCC 13.2.1 20230801] (/usr/bin/python)
jinja version = 3.1.2
libyaml = True
“”"

Before Galaxy NG this always worked. I mean “9.1.0+3.4.7” is already imported, that’s true. But how should I import otherwise? I can’t specify a tag e.g.

But besides that in general I can really just agree with The new Galaxy is completely broken . E.g. most of the versions of my roles just don’t match anymore with the tags I have in the Github repos: Galaxy NG

Then calling everything “role” like legacy is just strange (at least one feels like that using Galaxy NG). There are way more roles out there then collections and why create a collection if you’d only have one role in the collection? This is really telling all the role authors out there: “Go away! We don’t want you anymore!” At least that’s my impression right now.

The new UI looks like some retro pages from around year 2000 or so. Also why do I’ve to go to “Collections” to find my API token (needed to read the documentation to find it…)? Not very intuitive. At the end that belongs to the user settings/preferences IMHO.

Then the role names are not correct. E.g.
https://galaxy.ansible.com/ui/standalone/roles/githubixx/kubernetes-controller/
https://galaxy.ansible.com/ui/standalone/roles/githubixx/kubernetes-worker/
I renamed them a while ago to get rid of the “-” because “ansible-lint” was always complaining that this is not correct with the dash. But when I first started with that role names years ago it wasn’t an issue. Also here the version numbers are not correct.

I guess there are still way more issues out there. Today I only wanted to update my etcd role. But yeah, got lost somewhere

Sorry for the rant but I do all that in my free time and currently Galaxy NG is just consuming my spare time and there is nothing I get back. I don’t see a single plus with Galaxy NG as a role developer right now.

githubixx · October 19, 2023, 1:01pm

I know that roles are a thing of the past but is there a chance to make at least “ansible-galaxy role import” working again any time soon @galaxy ?I’ve quite some roles I want to update but they all fail with the same problem (generated a new token already as mentioned above):

https://github.com/githubixx/ansible-role-kubectl/actions/runs/6566961560/job/17838821405

Run ansible-galaxy role import --api-key *** $(echo githubixx/ansible-role-kubectl | cut -d/ -f1) $(echo githubixx/ansible-role-kubectl | cut -d/ -f2)
Successfully submitted import request 2052352144117301276862168712227772270

githubixx.kubectl 9.0.1+1.15.5 has already been imported
  File "/venv/lib64/python3.11/site-packages/pulpcore/tasking/tasks.py", line 66, in _execute_task
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/app/galaxy_ng/app/api/v1/tasks.py", line 145, in legacy_role_import
    raise Exception(msg)

or

https://github.com/githubixx/ansible-role-etcd/actions/runs/6551921619/job/17794582521

Run ansible-galaxy role import --api-key *** $(echo githubixx/ansible-role-etcd | cut -d/ -f1) $(echo githubixx/ansible-role-etcd | cut -d/ -f2)
Successfully submitted import request 2052238585650627843962377531833908658
running
githubixx.etcd 9.1.0+3.4.7 has already been imported
  File "/venv/lib64/python3.11/site-packages/pulpcore/tasking/tasks.py", line 66, in _execute_task
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/app/galaxy_ng/app/api/v1/tasks.py", line 145, in legacy_role_import
    raise Exception(msg)

I guess this will happen to every role I’d like to update. Regarding used Ansible and Ansible Galaxy version it uses the latest available:

Run pip3 install ansible-core
Collecting ansible-core
  Downloading ansible_core-2.15.5-py3-none-any.whl.metadata (7.0 kB)
Collecting jinja2>=3.0.0 (from ansible-core)
  Downloading Jinja2-3.1.2-py3-none-any.whl (133 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 133.1/133.1 kB 2.2 MB/s eta 0:00:00
Collecting PyYAML>=5.1 (from ansible-core)
  Downloading PyYAML-6.0.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.1 kB)
Collecting cryptography (from ansible-core)
  Downloading cryptography-41.0.4-cp37-abi3-manylinux_2_28_x86_64.whl.metadata (5.2 kB)
Collecting packaging (from ansible-core)
  Downloading packaging-23.2-py3-none-any.whl.metadata (3.2 kB)
Collecting resolvelib<1.1.0,>=0.5.3 (from ansible-core)
  Downloading resolvelib-1.0.1-py2.py3-none-any.whl (17 kB)
Collecting MarkupSafe>=2.0 (from jinja2>=3.0.0->ansible-core)
  Downloading MarkupSafe-2.1.3-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.0 kB)
Collecting cffi>=1.12 (from cryptography->ansible-core)
  Downloading cffi-1.16.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.5 kB)
Collecting pycparser (from cffi>=1.12->cryptography->ansible-core)
  Downloading pycparser-2.21-py2.py3-none-any.whl (118 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 118.7/118.7 kB 17.8 MB/s eta 0:00:00
Downloading ansible_core-2.15.5-py3-none-any.whl (2.2 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.2/2.2 MB 40.8 MB/s eta 0:00:00
Downloading PyYAML-6.0.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (757 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 757.7/757.7 kB 65.7 MB/s eta 0:00:00
Downloading cryptography-41.0.4-cp37-abi3-manylinux_2_28_x86_64.whl (4.4 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.4/4.4 MB 83.6 MB/s eta 0:00:00
Downloading packaging-23.2-py3-none-any.whl (53 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 53.0/53.0 kB 8.0 MB/s eta 0:00:00
Downloading cffi-1.16.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (464 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 464.8/464.8 kB 40.3 MB/s eta 0:00:00
Downloading MarkupSafe-2.1.3-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (28 kB)
Installing collected packages: resolvelib, PyYAML, pycparser, packaging, MarkupSafe, jinja2, cffi, cryptography, ansible-core
Successfully installed MarkupSafe-2.1.3 PyYAML-6.0.1 ansible-core-2.15.5 cffi-1.16.0 cryptography-41.0.4 jinja2-3.1.2 packaging-23.2 pycparser-2.21 resolvelib-1.0.1

Thanks!

githubixx · October 20, 2023, 6:10am

Looks like “ansible-galaxy role import --api-key” is working again. Thanks for whoever fixed that!

But now there are other problems. The “versions” are gone:

Now it states “master” after a new role import while the latest Github tag is “13.0.0+3.5.9” (see Tags · githubixx/ansible-role-etcd · GitHub) And as you can see in the screenshot it doesn’t show any versions but actually I’ve quite a few tags.

If I list my Galaxy namespace some roles are now showing up two times:

The old one is the one with the dash “-” in the role name and the newer one is the one with the underscore “_”. I changed the role name with version “23.0.0+1.27.5” as “ansible-lint” was always telling me that “-” is not allowed. This happened before Galaxy became NG. But since then I had a few more releases and the current one is “23.1.2+1.27.5”. In the role overview (the one with the “_”) the version is actually correct but it doesn’t show any previous versions:

But most probably that’s related to the error you see at the top of the screenshot: “Failed to load role versions”. Not sure if it’s already a problem that happened during import or it just a display bug.

So for newly imported roles the versions disappear and it tells you there is only a “master” branch which is not correct as I always tag my roles and with old Galaxy it always worked.

Besides that I’m also not able to update my profile anymore. Maybe because I’m no “super user”? :
superuser

Maybe someone can have a look. Thanks!

tannerjc · October 20, 2023, 8:31pm

Yep, this class of problem is on my radar and slated to be fixed. The mismatching of namespace.name/github_user and repository.name/github_repo was something we didn’t originally account for with this new v1 code.

L3D · October 21, 2023, 12:56am

I am happy to hear that it is still on someones radar

But yeah, it is still verry disapointing. Some strange stuff is happening with the galaxy-ng roles. Namespaces and imports don’t work properly anymore and so on.

And there is not even the option (jet) to remove all the own roles completly from galaxy and try again later or convert them to collections, because I don’t see an option to do that with my roles. And my 2-5 namespaces. (They random appear with or without case sensetive letters and stuff like that).

githubixx · October 24, 2023, 6:56pm

While “ansible-galaxy role import --api-key" first not worked, then worked and now it doesn’t work again:

Successfully submitted import request 2052961755339228647881627219293384474
running
githubixx.etcd master commit:86566ddaa42873ec66ecf7c761aa008bd498da05 has already been imported
  File "/venv/lib64/python3.11/site-packages/pulpcore/tasking/tasks.py", line 66, in _execute_task
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/app/galaxy_ng/app/api/v1/tasks.py", line 297, in legacy_role_import
    raise Exception(msg)

It now complains that “master” was already imported which in general is true (because it did import it with the last import - which in turn was a fault in doing so) but actually I never wanted to have “master” branch imported. The “versions” tab in the UI is now working again but now there is also the mentioned “master” version. How to get rid of that one?

Actually what would be most important that these “versions” stuff works. I mean personally I don’t care but for users this is pure evil if you rely on versioning. What’s currently in “master” version contains a few breaking changes and having a broken etcd cluster is a nightmare.

While the etcd role has only around 3800 downloads, my Wireguard role has around 150k. I’m currently totally frighted in touching anything in Galaxy because it most probably breaks something. I’m really thinking about deleting everything on Galaxy and just change any links to Github - but maybe even deletion wont work as according to the Galaxy UI I’m not the owner of my roles

I also currently have totally no idea what users are actually downloading if it comes to roles which formerly had a “-” in the name but where renamed to have a “_” instead because “-” was not allowed according to “ansible-lint”. I’m currently have both of them in my role list (as already mentioned in a previous post).

What really scares me is what happens to the users of my roles. If they did some updates the last few weeks they most probably have an - lets say - “interesting” state on their machines. And what happens if everything is fixed? I just can hope that it is not possible to download that “master” version…

Don’t get me wrong: I can only assume that there are people working in the background and do their best. But personally I feel totally helpless ATM. The current situation would really require an incident manager who communicates all the current states and is really watching all the issues and gives feedback on what’s worked on. But from an outsider view there is none. So I’ve currently no idea what is actually happening from all the problems I mentioned in a previous post. I can only assume that I need to re-post them every few days to get - hopefully - some attention. Posting something every few days and just stating that “people should be nice” doesn’t help. And it’s definitely not the task of the Galaxy developers to play the role of an incident manager. Communication is really everything in such a situation.

L3D · October 24, 2023, 8:19pm

I habe simmilar worrys than @githubixx .

Currently my discovery is, thant galaxy-ng is only able to import the lässt released Tag Form github. I created this github-action than will do the releasing for my roles:

And it looks like it is Wirkung for my l3d.gitea role.

githubixx · October 30, 2023, 1:43pm

So “ansible-galaxy role import …” is now working again Thanks for fixing that!

That means for me currently two issues are left if I’m right:

I’m not superuser of my namespace anymore
I’ve now roles with dash (“-” e.g. githubixx.kubernetes-controller) and with underscore (“_” e.g. " githubixx.kubernetes_controller"). The ones with underscore are the correct ones. The ones with dash are the ones with the old naming and which should not be used according to “ansible-lint”.

tannerjc · October 30, 2023, 1:49pm

@githubixx re “superuser” … If I look at the API data for the githubixx namespace, it lists the “githubixx” username as an owner of the namespace.

[jtanner@p1 ~]$ curl -s https://galaxy.ansible.com/api/v1/namespaces/2556/ | jq .summary_fields.owners
[
  {
    "id": 2563,
    "username": "githubixx"
  }
]

Is there something in the UI or some denied action you are trying to perform that makes you believe you are not the namespace owner?

Your username in galaxy is “githubixx” right?

githubixx · October 30, 2023, 2:05pm

@tannerjc I’m not really familiar with the Galaxy API TBH… I’m referring to the UI, yes. So if I open Galaxy NG user profile there is a slider at the very end under “User type”:

And it says “Not a super user”. And I can’t edit any fields like “First Name” e.g. I guess that I need to be super user to edit that fields. I’m also speculating that this would allow me to get rid of the duplicate roles that are listed with “_” and “-” in their names.

“Your username in galaxy is “githubixx” right?” → Yes, that right.

tannerjc · October 30, 2023, 2:12pm

Ah, I see. Nobody except sysadmins for galaxy should have the superuser flag set to true.

Control over namespaces in galaxy is handled via “ownership”. To see what namespaces you own, visit this link Galaxy NG

To delete roles, you have two options as the namespace owner.

ansible-galaxy role delete <github_user> <github_repo> this unfortunately fairly broad right now, so it would delete all roles matching those attributes. That behavior is probably expected when there are no duplicates.
curl -X DELETE -H 'Authorization: token <your-api-token>' https://galaxy.ansible.com/api/v1/roles/<roleid>/ … this lets you delete a specific role instead of all those matching the the user+repo.

githubixx · October 30, 2023, 8:03pm

@tannerjc Thanks a lot! It looks like that now everything is working again as expected!

Just in case somebody comes across with a similar problem. To get the “roleid” I listed all my roles with this command (of course you need to replace the value for “github_user”:

curl -s "https://galaxy.ansible.com/api/v1/roles/?github_user=githubixx&page_size=50" | jq '.results[] | {name: .name, id: .id}'

And then I was able to delete the roles with the underscore like described above with curl -v -X DELETE ... I got curl: (92) HTTP/2 stream 1 was not closed cleanly: PROTOCOL_ERROR (err 1) but the role was gone at least.

One final question (not directly related to the previous issues): Is it possible to “rename” a role and users are still able to find it? Originally my role was called githubixx.wireguard. At that time I had no role_name in meta/main.yml. Then something changed on Galaxy and suddenly the role was called githubixx.ansible_role_wireguard. I’d really like to have back the old name githubixx.wireguard but I was not brave enough to change role_name accordingly
But I read somewhere quite a while ago (can’t find anymore) that if a role name starts with ansible_role_ and you rename the role to a name without that prefix that it automatically returns the role name without that prefix if it exists. So if I now change role_name: ansible_role_wireguard to role_name: wireguard and a user still uses ansible-galaxy role install githubixx.ansible_role_wireguard it’d still work. Is this actually true or maybe I just dreamed of it?

tannerjc · October 30, 2023, 8:13pm

There is a facility in the backend code to strip “ansible-role-” from the name by default at import time, but still allow the --role-name argument (aka alternate_role_name) and the meta/main.yml to override that.

However, I’ve not come across any evidence that it works to install a role with “ansible_role_” prefixed on the name even though that is not the name.

(py3) [jtanner@corsair tmp]$ rm -rf ~/.ansible ; rm -rf testroles ; ansible-galaxy role install -s https://old-galaxy.ansible.com geerlingguy.docker
Starting galaxy role install process
- downloading role 'docker', owned by geerlingguy
- downloading role from https://github.com/geerlingguy/ansible-role-docker/archive/7.0.1.tar.gz
- extracting geerlingguy.docker to /home/jtanner/.ansible/roles/geerlingguy.docker
- geerlingguy.docker (7.0.1) was installed successfully

(py3) [jtanner@corsair tmp]$ rm -rf ~/.ansible ; rm -rf testroles ; ansible-galaxy role install -s https://old-galaxy.ansible.com geerlingguy.ansible_role_docker
Starting galaxy role install process
- downloading role 'ansible_role_docker', owned by geerlingguy
[WARNING]: - geerlingguy.ansible_role_docker was NOT installed successfully: - sorry, geerlingguy.ansible_role_docker was not found on https://old-galaxy.ansible.com/api.
ERROR! - you can use --ignore-errors to skip failed roles and finish processing the list.

(py3) [jtanner@corsair tmp]$ rm -rf ~/.ansible ; rm -rf testroles ; ansible-galaxy role install -s https://old-galaxy.ansible.com geerlingguy.ansible-role-docker
Starting galaxy role install process
- downloading role 'ansible-role-docker', owned by geerlingguy
[WARNING]: - geerlingguy.ansible-role-docker was NOT installed successfully: - sorry, geerlingguy.ansible-role-docker was not found on https://old-galaxy.ansible.com/api.
ERROR! - you can use --ignore-errors to skip failed roles and finish processing the list.

githubixx · October 30, 2023, 9:06pm

@tannerjc Many thanks for investigating!

system · November 29, 2023, 9:07pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.