Following the adoption of the new collection inclusion criteria, I propose discussing a roadmap to remove collections that no longer meet the criteria from the Ansible community package.
The current package includes 87 active collections. After applying the inclusion criteria, 17 collections qualify to stay. The remaining 70 are proposed for removal.
Each removed collection falls into at least one of:
Vendor-specific without being widely enough adopted to qualify for the infrastructure exception (unlike AWS, Azure, GCP).
Domain-specific or niche, not broadly applicable across industries.
Application-layer (databases, message brokers, monitoring tools) rather than foundational infrastructure.
Deprecation timeline
Announce deprecation: Ansible community package 14 (current release).
Remove from package: Ansible community package 18 (~2 years / 4 major releases later).
This gives users roughly two years to adjust CI/CD pipelines and add explicit ansible-galaxy collection install steps. The collections will remain available on Galaxy.
What this means for users
Removed collections stay on Galaxy and can be installed individually.
Existing playbooks and roles continue to work after installing the needed collections explicitly.
No functionality is lost, only the bundling changes.
What we need to decide now
To resolve this topic we need to make decisions on the following questions in order:
Do we support trimming down the package? If yes, proceed to the next questions.
Do we keep ansible-core + only the suggested collections? Should we adjust the list (add/remove anything)?
Do we use the suggested deprecation timeline?
Please share your thoughts on the above questions in comments. Wording suggestions are appreciated in the PR.
I believe we should not only “announce” but also advertise periodically (monthly? 3 months? 6months ~ at major versions’ releases?), reminding users about the deprecation.
YES!
Not something that I would defend fiercely, but maybe add open source on-prem virtualisation platforms, like Openstack and Proxmox.
Since this is a very large change to the Ansible community package, I propose to slow this discussion down and not decide anything until very late in the Ansible 14 release cycle, or even just for 15.0.0 (or one of its pre-releases). This gives more interested parties (users, OS packagers, affected collection developers) time to take part in the discussion (if they want to).
The new criteria does not explicitly deny application layer, nor does it explicitly say only infrastructure layer
Limiting the collections to just infrastructure deters non-systems people from trying Ansible (for example, a DBA or developer), as well as limits a users ability to see the full “chain” of automation for their full stack.
Surely a database or monitoring tool is considered “foundational”
@russoz@felixfontein@mikemorency great feedback, thank you! I agree with all the statements. Let’s collect more feedback and then I’ll try to incorporate overlapping opinions into the PR
I’ve been thinking a lot about what to include, or how to avoid ending up with basically the same package size as before
I agree that some more collections (proxmox/openstack for virtualization, mysql/mariadb/postgres for databases, … no idea what for monitoring, …) should likely be part of the package. I’m also wondering about network collections. I think someone mentioned that among network infra people it is quite common to install the Ansible community package since it contains a lot of networking stuff. The proposal above removes it all, despite networking being quite foundational (most servers would have nothing to do without it ). But with this chain of arguments, you end up with good arguments to keep most things…
How about the (in the past, mentioned more than once) scenario of creating multiple Ansible community packages? Then we could have ansible-network that contains ansible-core + ansible.netcommon + a set of network connections, ansible-cloud that contains ansible-core + amazon/community.aws, azure.azcollection, google.could, etc., and ansible-virtualization that contains …, ansible-monitoring that contains …, and so on. That would provide a smaller list of Ansible community packages that are tailored for different use-cases. Most people would install two or more of these.
(There shouldn’t be too many, otherwise we could just package collections as PyPI packages… Also the more there are, the more work there is for us…)
(Also it’s too bad that the names ansible-core and ansible-base as PyPI packages are already taken, they could be used for more foundational bundles )
One of the reasons for stopping the expansion is that it’s unworkable long term. That much was clear.
I think we really need to look at what our long term goal is, and I agree strongly with @felixfontein here: there’s no rush. It’s better that we make the right decision here and miss the cut-off for the next major release cycle, than pressure ourselves to meet a deadline, just for the sake of meeting the deadline.
My personal suggestions would be that we either
Accept some of the base application layers (Think old-school RHCE), or
We cut it back further than this proposal (strip out Public Cloud and Kubernetes) and have a strategy for grouping some of the collections (Cloud, Apps, Network, Database, Monitoring), which should still be mostly the “big” players in each area to avoid bloat.
If we’re building a way to group the collections, we could even thing about offering the “kitchen-sink” group, which is all of the other groups.
This is the same problem we had in core, which ‘collections’ were meant to address, it did work well for core, but the Ansible community package, which was initially meant as a stop gap measure, was given more life by the community, but also perpetuated this issue.
I suggest asking the RH community team to compile a list of the top 20 collections and compare that to this list. Docsite stats would help here just to see if it matches or diverges in big ways. Not to say the lists need to match, but if they diverge, it gives y’all information to help the planning and communication of this change.
Meta collections are a neat idea and likely much easier for the community to maintain vs mini- ACP for networking, databases, etc.
And as someone who doesn’t actually use ACP, I’ll ask the silly question (and point me to a thread if this was already covered) - why not stop ACP altogether? Going down to 17 or so is drastic, so if we go that far, why not all the way and just leave Ansible-core?
Removing 70 from 87 collections somehow feels like overdoing things. I agree that we should remove some niche collections, but what are they? I guess we’ll find some, but probably not 70. Maybe we should concentrate on the low hanging fruit, which sounds more like possibly 10 or 15 collections. At least to start with. Removing 80% of the collections would be a huge change.
BTW I personally would like to keep the VMware collections. Although Broadcom goes out of its way to change this, I think VCF / vSphere / ESX still falls under “widely adopted”. At least in enterprise environments. Apropos of nothing: You want to keep community.libvirt, but this has some 3.7 million downloads on galaxy while community.vmware has some some 23 millions. Maybe libvirt is more widely adopted than the VMware solution, but does it also mean there are more community.libvirt users than there are community.vmware / vmware.vmware / vmware.vmware_rest users? I think we have not enough data there.
DISCLAIMER: I’m the maintainer of community.vmware, and I’m also involved in vmware.vmware and vmware.vmware_rest. But that’s not why I want to keep the collections in ACP. The reason is that we use them a lot via ACP. Selfish? Definitely. But at least I tell you, no hidden agenda or anything. I’m putting my cards on the table.
You can just announce the 70 you propose to remove, add them back as people start yelling. This will probably tell you which collections are actually in use.
But I think the low hanging fruit approach is less stressful.
Another solution would to add telemetry and gather usage, but that opens up other concerns, like privacy and now you have to maintain a collection service.
I wouldn’t like to open this can of worms. 1) I think it would be complicated and we would need all of the 87 currently included collections to cooperate and implement this, wouldn’t we? 2) Telemetry would be really helpful, but I myself value privacy higher.
This might have some issues in practice due to everything being installed under the ansible_collections directory. For example the ansible package on PyPI will store it’s collections inside the ansible_collections “package” so that Ansible will find them:
Other collection like packages will also need to store their collections in ansible_collections/ and while I believe pip will merge them other tools may not be so friendly. The uninstall of a collection will also be tricky if we start getting the same collections in different packages as uninstalling one will most likely also remove it for other collections that also relied on and shipped their own. Not to mention version conflicts that the Python resolver has no idea about.
