Ansible Galaxy’s role import enhancements and fixes for the new year

[tannerjc (James Tanner)]

Ansible Galaxy’s role import enhancements and fixes for the new year.

Import logging …

Remember when you could see a nice formatted list of messages when your roles were imported? We’ve brought that back (see Role import logging by jctanner · Pull Request #1959 · ansible/galaxy_ng · GitHub)! You may have already noticed, but when you run “ansible-galaxy role import” it displays colorized messages about what the backend is doing with the role (where it’s cloning from, what lint tests pass or fail, etc). These logs are also displayed in the UI when you visit any role and click on “import log”. We implemented this back in November, but it took a little longer to get the UI changes rolled out.

Defining a role’s namespace and name …

Role import “magic”:
For those of you who had roles in the old galaxy with namespaces and names that differ from their related github user and github repo, we implementented some “magic” to route the import to the most “correct” role by namespace and name. While this feature made it possible to keep importing new versions of those roles, it still didn’t address the underlying issue that authors had no way to specify which namespace and name they wanted.

New standard keys in the meta/main.yml file:
namespace & role_name in meta/main.yml: You can now specify these keys in your meta/main.yml. This means when you’re importing a role, Galaxy will try to slot it into the namespace you’ve mentioned with the role_name you’ve specified. This makes the role less “portable” in terms of being able to import it into multiple different namespace.name configurations, but it does make the import much more predicatable for those who have namespaces or role names that are different from the github user and github repo. This feature is live in production now and ready to use. Please send us feedback if you have any!

We’ll still keep the “magic” finder code for a while, but it’s only going to trigger if the author hasn’t defined any overrides to the namespace and name.

New API changes for namespace and name:

Remember when we could use alternate-role-name in the Galaxy API? It was a nifty feature, but it got dropped a while back (see ansible/galaxy@1d1519f). Since then, the Galaxy gang has been busy bees, adding namespace support in the galaxy-importer schema (ansible/galaxy-importer@98e0931) and slipping in role_name into the legacy role schema (ansible/galaxy-importer@1927cd6).

As part of support legacy role import overrides for namespace and name. by jctanner · Pull Request #2011 · ansible/galaxy_ng · GitHub, we’ve added support in the API for alternate_namespace_name and alternate_role_name in the POST data for importing a role. If provided, the backend will use these values as the highest order of precedence for determining the namespace+name for the role, regardless of the github user or github repo the role lives in. We don’t suspect many people will use the API directly for importing roles, but the option is avaible. We do plan to use these new parameters to support future enhancements to the UI that allow role authors to import directly into a namespace and or a specific role name. That feature should get us very close to satisfying one of the biggest requests from the community so far.

UPDATE 2024-01-15

We’ve built and deployed role upload components to the galaxy UI. You can now import roles from the UI and take advantage of the namespace and name override fields …

[githubixx]

Cloud it be that these changes broke ansible-galaxy role import ... again? I was wondering why my githubixx.kubernetes_controller role wasn’t updated until I figured out that there is now (again) a new role githubixx.kubernetes-controller. That’s actually interesting as - are not allowed in role names. ansible-lint completely freaks out if you do that My last import was on Dec 13 and that worked as expected.

This is the Github Action log:

Run ansible-galaxy role import --api-key *** $(echo githubixx/ansible-role-kubernetes-controller | cut -d/ -f1) $(echo githubixx/ansible-role-kubernetes-controller | cut -d/ -f2)
  
Successfully submitted import request 2060918917000330747058366163679919697
Starting import: task_id=2060918917000330747058366163679919697, pulp_id=018ceb25-cc9a-74bb-a68f-8fa668173e51

==== PARAMETERS ====
importer username: githubixx
matched user: githubixx id:2563
github_user: githubixx
github_repo: ansible-role-kubernetes-controller
github_reference: None
alternate_clone_url: None
alternate_namespace_name: None
alternate_role_name: kubernetes-controller

==== CHECK FOR MATCHING ROLE(S) ====
user:githubixx repo:ansible-role-kubernetes-controller matched existing role githubixx.kubernetes_controller id:32487

===== CLONING REPO =====
cloning https://github.com/githubixx/ansible-role-kubernetes-controller ...

===== GIT ATTRIBUTES =====
github_reference(branch): master
github_commit: 48a7477f08fbbf0214b633fd801de8885f9b1a93
github_commit_message: 23.0.0+1.28.5 (#55)

* upate k8s_release to 1.28.5 / extend enable-admission-plugins list of kube-apiserver

* update CHANGELOG and README

* Molecule: Change IP addresses
github_commit_date: 2024-01-08T23:09:49+01:00

===== LOADING ROLE =====
Importing with galaxy-importer 0.4.18
Determined role name to be kubernetes_controller
Linting role kubernetes_controller via ansible-lint...
ansible-role-kubernetes-controller/molecule/default/prepare.yml:53:7: syntax-check[specific]: couldn't resolve module/action 'ansible.posix.synchronize'. This often indicates a misspelling, missing collection, or incorrect module path.
ansible-role-kubernetes-controller/molecule/default/verify.yml:8:7: syntax-check[specific]: couldn't resolve module/action 'kubernetes.core.k8s_info'. This often indicates a misspelling, missing collection, or incorrect module path.
...ansible-lint run complete
Legacy role loading complete

===== PROCESSING LOADER RESULTS ====
overriding namespace name via metadata: githubixx
enumerated role name kubernetes-controller
created new role id:37622 githubixx.kubernetes-controller

I guess alternate_role_name: kubernetes-controller is wrong and it’s also not in the metadata. meta/main.yml looks like this:

---
galaxy_info:
  author: Robert Wimmer
  description: Installs the Kubernetes API server, scheduler and controller manager.
  license: GPLv3
  min_ansible_version: "2.9"
  role_name: kubernetes_controller
  namespace: githubixx
  platforms:
    - name: Ubuntu
      versions:
        - "focal"
        - "jammy"
  galaxy_tags:
    - kubernetes
    - scheduler
    - api
    - controller
    - ha

The role_name also is set to kubernetes_controller (with “_” and not “-”).

So this is the new (and wrongly named) role:
https://galaxy.ansible.com/ui/standalone/roles/githubixx/kubernetes-controller/

But it should be imported here:
https://galaxy.ansible.com/ui/standalone/roles/githubixx/kubernetes_controller/

Thanks for checking!

[tannerjc (James Tanner)]

I see the issue …

ansible/ansible/blob/devel/lib/ansible/galaxy/api.py#L485-L486

Core is injecting the alternate_role_name parameter based on the name of the repository. Based on Support user defined role names in role metadata only · ansible/galaxy@1d1519f · GitHub that hasn’t actually done anything in old-galaxy since 2018. However, in new galaxy with the patch I mention in this announcement it has been restored.

In your case, your github action workflow should change to something like …

ansible-galaxy role import -vvvvvvvv --role-name=$(echo ${{ github.repository }} | cut -d/ -f2 | sed 's/ansible-role-//' | sed 's/-/_/') $(echo ${{ github.repository }} | cut -d/ -f1) $(echo ${{ github.repository }} | cut -d/ -f2)

Passing the --role-name will stop core from trying to do the wrong thing.

[jburel (jean-marie burel)]

Facing a similar issue when publishing a new version of an existing role via GitHub action.

Do you suggest to apply the work around using --role-name?
Note that this broke all the existing Github actions used to publish to Galaxy e.g. ansible-actions/ansible-galaxy-action since none of the existing actions offers the ability to specify the role-name as a parameter. So users will be forced to use ansible-galaxy role import... command directly.

Or do you suggest to rename the repository to use underscore instead of dash?

Thanks for clarifying that critical issue

[tannerjc (James Tanner)]

@jburel in my personal opinion, use --role-name. The long term fix (again in my opinion) is for core to remove the code I mentioned in my previous response.

Changing the repository name would also theoretically work, but I sense that not many people would actually want to do that.

[jburel (jean-marie burel)]

Thanks for the answer

[shertel]

This is on my to-do list to fix. Is there any special behavior when uploading a role via the web ui anymore if the role starts with ansible-role-? That was the reason it was added in Allow setting alternate_role_name for galaxy CLI (#17418) · ansible/ansible@bd9ca5e · GitHub,

Also, the web ui is smart and imports
“openstack-infra/ansible-role-puppet” as openstack-infra.puppet … but
the CLI imports it as openstack-infra.ansible-role-puppet. Add that
filtering as well.

[tannerjc (James Tanner)]

No special behavior anymore. The code to truncate ansible-role- from the repo name no longer exists in the backend.

[githubixx]

Thanks a lot @tannerjc ! That worked for me.

[staticdev]

This is good. I think if you don’t specify a role_name on either meta/main.yml or ansible-galaxy, it doesn’t need to guess and use what you passed as attribute: github_repo.

From usage:

usage: ansible-galaxy role import [-h] [-s API_SERVER] [--token API_KEY] [-c] [--timeout TIMEOUT] [-v] [--no-wait] [--branch REFERENCE] [--role-name ROLE_NAME] [--status] github_user github_repo

BUT, the problem is, if you don’t specify --role-name on ansible-galaxy and specify on meta/main.yml it should use the name specified. This is not the case and it creates unnecessary extra breaking change. I propose to have this behavior.

In my case I call ansible-galaxy with:

ansible-galaxy role import --api-key *** staticdev ansible-role-python-developer

But I define on meta/main.yml as python_developer. But it is being imported as python-developer. This is completely counter-intuitive, I am against having to add additional parameter on the CLI since it is already specified on metadata.

[tannerjc (James Tanner)]

@staticdev I agree with you. The core team appears to want to make that change here [ansible-galaxy] Remove role name conversion when the repo startswith 'ansible-role' by s-hertel · Pull Request #82508 · ansible/ansible · GitHub but we’ll have a long while before every supported version of core out in the wild has that patch since it probably won’t be backported.

We’ll have to come up with some way to mitigate the confusion in the meantime.

[dbrennand]

But I define on meta/main.yml as python_developer. But it is being imported as python-developer. This is completely counter-intuitive, I am against having to add additional parameter on the CLI since it is already specified on metadata.

I agree with @staticdev here. I’ve also just experienced the same problem. My role’s meta/main.yml with:

galaxy_info:
  role_name: caddy_docker
  ...

Ended up being imported as caddy-docker, which caused quite some confusion for me . It does seem a bit counter-intuitive to specify --role-name on the CLI when the details are already defined in meta/main.yml

[dbrennand]

For those using GitHub Actions, I’m using the following workaround. Posting here in case this helps anyone else who experiences a similar issue.

[shertel]

Backports have been merged for [ansible-galaxy] Remove role name conversion when the repo startswith 'ansible-role' by s-hertel · Pull Request #82508 · ansible/ansible · GitHub, so the issue with repositories that start with the prefix ansible-role will be fixed in 2.15.9 and 2.16.3 (rc1 scheduled for Jan 22, GA on Jan 29). The ansible-role-name → name conversion has been removed, and the galaxy_info role_name or the --role-name option can override the repository as the role name. The only workaround for older versions of anible-core is to use the --role-name option.