# Galaxy importer: collection dependency handling **URL:** \ **Category:** Project Discussions\ **Tags:** vmware, dependencies, aws, collections, galaxy-ng\ **Created:** [January 27, 2024, 12:48pm UTC](https://forum.ansible.com/t/galaxy-importer-collection-dependency-handling/3496 "2024-01-27T12:48:27Z")\ **Posts on this page:** 15\ **Page:** 1
**Author:** ![felixfontein](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/felixfontein/32/7530_2.png) [@felixfontein](https://forum.ansible.com/u/felixfontein)\ **Post date:** [January 27, 2024, 12:48pm UTC](https://forum.ansible.com/t/galaxy-importer-collection-dependency-handling/3496/1 "2024-01-27T12:48:27Z")
I noticed that apparently galaxy-importer does not care about collection dependencies that have been installed before running it, and it itself does no attempt at installing collection dependencies, and apparently also doesn’t do that when publishing collections to [https://galaxy.ansible.com/](https://galaxy.ansible.com/). This has a serious drawback: some collections use docs fragments from collections they depend on! Since galaxy-importer does not use any collection except the one to import, this means that any module / plugin whose documentation depends on another collection has no documentation rendered in the UI at [https://galaxy.ansible.com](https://galaxy.ansible.com)! Examples: - The inventory plugins in community.docker use the docs fragments from community.library\_inventory\_filtering\_v1. The result is that the docs for the inventory plugins are empty: - [Ansible Galaxy](https://galaxy.ansible.com/ui/repo/published/community/docker/content/inventory/docker_containers/) - [Ansible Galaxy](https://galaxy.ansible.com/ui/repo/published/community/docker/content/inventory/docker_machine/) - [Ansible Galaxy](https://galaxy.ansible.com/ui/repo/published/community/docker/content/inventory/docker_swarm/) - community.aws uses docs fragments from amazon.aws for almost all modules and plugins (or even all of them, I didn’t check everything). That means that all module/plugin docs are missing, for example: - [Ansible Galaxy](https://galaxy.ansible.com/ui/repo/published/community/aws/content/module/accessanalyzer_validate_policy_info/) - [Ansible Galaxy](https://galaxy.ansible.com/ui/repo/published/community/aws/content/module/acm_certificate/) - [Ansible Galaxy](https://galaxy.ansible.com/ui/repo/published/community/aws/content/module/acm_certificate_info/) - [Ansible Galaxy](https://galaxy.ansible.com/ui/repo/published/community/aws/content/module/api_gateway/) - [Ansible Galaxy](https://galaxy.ansible.com/ui/repo/published/community/aws/content/module/api_gateway_domain/) (just clicking the first few). I think this is something that should get fixed. CC @tannerjc since you’ve looked at the docs loading code in galaxy-importer recently. ---
**Author:** ![felixfontein](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/felixfontein/32/7530_2.png) [@felixfontein](https://forum.ansible.com/u/felixfontein)\ **Post date:** [January 27, 2024, 12:50pm UTC](https://forum.ansible.com/t/galaxy-importer-collection-dependency-handling/3496/2 "2024-01-27T12:50:39Z")
(And yes, I do understand that this isn’t easy to fix 🙂 I mainly noticed when I was trying to get [GitHub - ansible-community/github-action-test-galaxy-import: GitHub Action for testing Galaxy import of an Ansible collection](https://github.com/ansible-community/github-action-test-galaxy-import/) to actually consider dependencies. I tried to look into how this was done on Zuul for community.aws, in the end realizing that they simply [disabled the ansible-doc part of the import](https://github.com/ansible-collections/community.aws/blob/main/tests/galaxy-importer.cfg), and then digging into the galaxy-importer code, to find out that it has no mechanism to use any collection dependencies, no matter whether they have been installed before or not.) ---
**Author:** ![felixfontein](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/felixfontein/32/7530_2.png) [@felixfontein](https://forum.ansible.com/u/felixfontein)\ **Post date:** [February 16, 2024, 6:45am UTC](https://forum.ansible.com/t/galaxy-importer-collection-dependency-handling/3496/3 "2024-02-16T06:45:06Z")
This also affects other tests run by the importer, like ansible-lint, see [Ansible-galaxy lint unexpected warnings](http://forum.ansible.com/t/ansible-galaxy-lint-unexpected-warnings/3825). ---
**Author:** ![felixfontein](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/felixfontein/32/7530_2.png) [@felixfontein](https://forum.ansible.com/u/felixfontein)\ **Post date:** [May 5, 2024, 7:38pm UTC](https://forum.ansible.com/t/galaxy-importer-collection-dependency-handling/3496/4 "2024-05-05T19:38:16Z")
Pinging [@galaxy](https://forum.ansible.com/groups/galaxy) in case you didn’t see this so far. I think this is rather serious since it breaks documentation rendering for all collections that use docs fragments from other collections. ---
**Author:** ![mariolenz](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/mariolenz/32/108_2.png) [@mariolenz](https://forum.ansible.com/u/mariolenz)\ **Post date:** [September 20, 2025, 11:54am UTC](https://forum.ansible.com/t/galaxy-importer-collection-dependency-handling/3496/5 "2025-09-20T11:54:34Z")
I’ve started to re-use code from `vmware.vmware` in `community.vmware` 6.0.0 and [run into this](https://galaxy.ansible.com/ui/repo/published/community/vmware/import-log/?version=6.0.0), too. This error isn’t really great, but it looks like this also leads to missing documentation on galaxy (see [here](https://galaxy.ansible.com/ui/repo/published/community/vmware/content/module/vcenter_domain_user_group_info/?version=6.0.0) for an example). And this is really problematic. `community.aws` seems to have the same problem, see [here](https://galaxy.ansible.com/ui/repo/published/community/aws/import-log/?version=10.0.0) and [here](https://galaxy.ansible.com/ui/repo/published/community/aws/content/module/accessanalyzer_validate_policy_info/?version=10.0.0) for examples. But @felixfontein already mentioned this. However, there are possibly more collections affected that I don’t know about. Actually, I think this not only possible but probable. [@galaxy](https://forum.ansible.com/groups/galaxy) Do you have any plans to fix this? And if not, why not? This looks like a real problem to me. ---
**Author:** ![mariolenz](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/mariolenz/32/108_2.png) [@mariolenz](https://forum.ansible.com/u/mariolenz)\ **Post date:** [September 20, 2025, 12:57pm UTC](https://forum.ansible.com/t/galaxy-importer-collection-dependency-handling/3496/6 "2025-09-20T12:57:00Z")
[@AWS](https://forum.ansible.com/groups/aws) Since `community.aws` is also affected, I wonder what you think about this. ---
**Author:** ![felixfontein](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/felixfontein/32/7530_2.png) [@felixfontein](https://forum.ansible.com/u/felixfontein)\ **Post date:** [September 20, 2025, 1:12pm UTC](https://forum.ansible.com/t/galaxy-importer-collection-dependency-handling/3496/7 "2025-09-20T13:12:01Z")
In the realm of Supported collections (with the capital S), ansible.netcommon also has two doc fragments that might be used by other Supported collections, whose docs will also not be shown correctly on Galaxy due to this bug. (Edit: I did check the collections included in Ansible 12, none of them use these doc fragments except ansible.netcommon itself.) ---
**Author:** ![tremble](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/tremble/32/247_2.png) [@tremble](https://forum.ansible.com/u/tremble)\ **Post date:** [September 22, 2025, 8:38am UTC](https://forum.ansible.com/t/galaxy-importer-collection-dependency-handling/3496/8 "2025-09-22T08:38:59Z")
Personally, I agree this is a bug that would hopefully get fixed. I did some digging when the “galaxy-importer” tests were first added to Zuul, and it’s more frustrating than galaxy-importer “not supporting” the installation of dependencies, it (at least at the time) explicitly runs in a self-contained context and ignores any dependencies that might have already been installed, so we couldn’t even work around the issue by explicitly installing the dependency as part of the setup for the test. Hence just disabling the ansible-doc pieces and moving to the [ansible-community/github-docs-build](https://github.com/ansible-community/github-docs-build/blob/main/.github/workflows/_shared-docs-build-pr.yml) GitHub action. Copying the fragments over from amazon.aws into community.aws would be problematic since the actual behaviour (including common parameters) is coming from amazon.aws and not community.aws ---
**Author:** ![mariolenz](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/mariolenz/32/108_2.png) [@mariolenz](https://forum.ansible.com/u/mariolenz)\ **Post date:** [September 22, 2025, 3:27pm UTC](https://forum.ansible.com/t/galaxy-importer-collection-dependency-handling/3496/9 "2025-09-22T15:27:48Z")
> [@tremble](#): > > Copying the fragments over from amazon.aws into community.aws would be problematic since the actual behaviour (including common parameters) is coming from amazon.aws and not community.aws I’m not sure if I understand. If the actual behavior and common parameters come from amazon.aws, the docs fragments should also. This way, both would automatically be aligned. Oh, wait! Let’s say c.a supports a.a X and later. If there’s a new a.a release Y with changes to common parameters, it’s not clear what docs fragments are the correct ones: Those from a.a X or a.a Y? After all, it depends on the a.a version people have installed. Is this what you’re meaning? ---
**Author:** ![sivel](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/sivel/32/1198_2.png) [@sivel](https://forum.ansible.com/u/sivel)\ **Post date:** [September 22, 2025, 3:37pm UTC](https://forum.ansible.com/t/galaxy-importer-collection-dependency-handling/3496/10 "2025-09-22T15:37:14Z")
I had commented somewhere else on why this might not be a feasible task, although I can understand why it wouldn’t be assumed to be complicated from just the community galaxy perspective, however… I foresee both technical hurdles and potentially security related hurdles to overcome. I’d have to defer to much more in depth evaluations from the galaxy team to understand all of the implications. Although, for one, galaxy-importer would have to write an ansible.cfg, of which it doesn’t have the context for, so it would be dependent on being provided that context from galaxy\_ng, creating some complex circular dependencies. Then it would have to run ansible-galaxy to install the deps. ansible-galaxy CLI is the owner of the dependency resolution mechanism, so it would also create additional dependencies. As such, I don’t see a way that galaxy\_ng could reasonably be expected on it’s own to just suck in the appropriate tarball of the dep. So galaxy\_ng would end up invoking galaxy-importer, to invoke ansible-galaxy, to call galaxy\_ng. There are likely to be hurdles with this approach from many common proxy configurations, in which the environment running ansible-galaxy may not be able to directly consume the galaxy\_ng APIs due to network security complexities. Also imagine the automation hub, or onprem hub experiences where there is the potential to have any arbitrary number of “repos”. Which repo would we install the deps from? Amongst like I mention, galaxy-importer doesn’t have the context of the repos available, so galaxy\_ng would have to provide that to importer in some way. Which also comes with RBAC considerations too, in which galaxy\_ng also doesn’t have a way to bypass it’s own security restrictions to be able to allow ansible-galaxy to make API calls. ---
**Author:** ![mariolenz](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/mariolenz/32/108_2.png) [@mariolenz](https://forum.ansible.com/u/mariolenz)\ **Post date:** [September 22, 2025, 3:50pm UTC](https://forum.ansible.com/t/galaxy-importer-collection-dependency-handling/3496/11 "2025-09-22T15:50:32Z")
@sivel Would it be possible to have a better documentation on galaxy? For example, [this](https://galaxy.ansible.com/ui/repo/published/community/vmware/content/module/vcenter_folder/?version=6.0.0) is just empty. This looks like there’s no documentation at all, but this is not true. What I mean is: Show the documentation that is not from another collection, and mention that other parameters come from another collection and are therefore not shown? Or at least mention that the documentation cannot be shown / rendered because it depends on other collection(s)? ---
**Author:** ![tremble](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/tremble/32/247_2.png) [@tremble](https://forum.ansible.com/u/tremble)\ **Post date:** [September 22, 2025, 4:03pm UTC](https://forum.ansible.com/t/galaxy-importer-collection-dependency-handling/3496/12 "2025-09-22T16:03:48Z")
> [@mariolenz](#): > > Oh, wait! Let’s say c.a supports a.a X and later. If there’s a new a.a release Y with changes to common parameters, it’s not clear what docs fragments are the correct ones: Those from a.a X or a.a Y? After all, it depends on the a.a version people have installed. Is this what you’re meaning? The correct fragments are those from the version of a.a you’re using. The “common” code in amazon.aws is usually directly related to configuring connections to the backend API (for example passing AuthN/AuthZ tokens), we have some code which will automatically add the relevant common parameters like “access\_key” (~username) and “secret\_key” (~password) to the argument\_spec, before then passing it down through to AnsibleModule. In theory, if Amazon added a new authentication mechanism, and we added support with amazon.aws 20.0.0 through some new parameters, you could actually use community.aws 19.0.0 with amazon.aws 20.0.0 and it would work and the community.aws modules would happily accept the extra parameters… ---
**Author:** ![felixfontein](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/felixfontein/32/7530_2.png) [@felixfontein](https://forum.ansible.com/u/felixfontein)\ **Post date:** [September 22, 2025, 4:45pm UTC](https://forum.ansible.com/t/galaxy-importer-collection-dependency-handling/3496/13 "2025-09-22T16:45:02Z")
> [@sivel](#): > > I had commented somewhere else on why this might not be a feasible task, although I can understand why it wouldn’t be assumed to be complicated from just the community galaxy perspective, however… I definitely don’t think this is easy to fix, it definitely needs a lot of architectural decisions (and insight) to come up with a good way to do this. I’m mostly annoyed that there has been **zero feedback** from anyone with more knowledge than the community until now. At least a “we see it is a problem and discussed it, but it’s far from trivial and we currently do not prioritize it” would have been nice. Regarding dependency resolution: it might be a good idea to make it possible to use ansible-galaxy (the CLI tool)'s resolution algorithm available (at least only for the galaxy\_ng backend) so it can use the resolution algorithm together with its database (and potential outside sources, depending on how galaxy\_ng is configured) to figure out the dependent collections that are needed, and provide their tarballs to galaxy-importer. Which can then extract all of them and run the import process as before. This would allow galaxy-importer to not need to be able to resolve or download any dependencies, and make it easy for galaxy\_ng to figure out what’s needed to provide to galaxy-importer. This obviously is still not trivial to implement and needs more thoughts and design etc., but sounds doable. > [@sivel](#): > > Also imagine the automation hub, or onprem hub experiences where there is the potential to have any arbitrary number of “repos”. Which repo would we install the deps from? Amongst like I mention, galaxy-importer doesn’t have the context of the repos available, so galaxy\_ng would have to provide that to importer in some way. Which also comes with RBAC considerations too, in which galaxy\_ng also doesn’t have a way to bypass it’s own security restrictions to be able to allow ansible-galaxy to make API calls. IIRC Galaxy doesn’t allow to upload collections where it cannot find the dependencies for. (Or has that requirement been lifted for galaxy\_ng? I think the last time I accidentally tried this was in 2020…) So somehow it does know where dependencies should be available from. I’ve never used AH/galaxy\_ng on-prem and don’t know about the “repo” feature, so I don’t know what exactly this means (besides some guesstimates from what you wrote) and can’t say anything about whether this is a big problem or not… But a solution that works in many cases would already be a lot better than the current one (which ignores dependencies completely). ---
**Author:** ![felixfontein](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/felixfontein/32/7530_2.png) [@felixfontein](https://forum.ansible.com/u/felixfontein)\ **Post date:** [September 22, 2025, 4:46pm UTC](https://forum.ansible.com/t/galaxy-importer-collection-dependency-handling/3496/14 "2025-09-22T16:46:37Z")
(Also there has been the implication that thanks to galaxy\_ng’s doc viewing capabilities, no separate docsite is needed anymore for collections. These claims likely didn’t come from folks who knew the problems this has, but considering that more and more users seem to use the Galaxy docs over [docs.ansible.com](http://docs.ansible.com) this is becoming a bigger problem.) ---
**Author:** ![mariolenz](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/mariolenz/32/108_2.png) [@mariolenz](https://forum.ansible.com/u/mariolenz)\ **Post date:** [September 22, 2025, 5:49pm UTC](https://forum.ansible.com/t/galaxy-importer-collection-dependency-handling/3496/15 "2025-09-22T17:49:34Z")
> [@felixfontein](#): > > Also there has been the implication that thanks to galaxy\_ng’s doc viewing capabilities, no separate docsite is needed anymore for collections. Personally, I’ve started to point people who are looking for documentation to galaxy. I wasn’t aware about those problems until I stumbled upon them a couple of days ago :-/ I might have seen those problems before and just forgot. But anyway, the current situation is somewhat unfortunate…