For people who don’t know, I’m Andrei, a Steering Committee member and a person who reviews all Ansible community package inclusion requests. To learn how the reviews are done, see ansible-inclusion/README.md.
In the susunola.tencentcloud inclusion thread, we see the first concrete case of what I think is a coming pattern: a collection with 1000+ plugins/modules, largely unreadable generated documentation, and 500+ sanity-test ignores to mask bugs and inclusion checklist violations and to make the CI green.
We -Steering Committee- didn’t build the inclusion workflow and policies around the AI-driven submission pace and volume that are built in a way to superficially pass the requirements.
Reviewer bandwidth was already a concern back in the community-topics repo discussion in 2022, before this kind of volume was even possible.
the package was originally meant to be transitional and fade away as collections matured, not grow indefinitely (it went from 81 to 99 collections between 2.10 and v5 alone)
“raise the bar” versus “curate by usefulness × popularity ÷ size”
the real onboarding/docs value the bundling provides
So my concerns and main points are:
Do we really need to proceed within the current scheme bloating the package and docs.ansible.com with content I’d expect to be niche?
Reviewer bandwidth (“Use AI to review” kind of suggestions isn’t a good solution to this specific problem IMO).
It’s still not a problem to download any collections you need.
This topic is about the ansible community package that users install with pip install ansible. Anybody can build any alternative one and publish it on PyPI whether it’s ansible-databases, ansible-cloud, or ansible-kitchensink accepting whatever collections they want.
Introducing new requirements or forbidding AI-generated collections is not going to work IMO.
My proposal: To tackle the problem at the core, trim the community package down to ansible-core plus a set of collections explicitly selected by the Steering Committee. The specific set of collections and criteria aren’t to be decided in this topic. The goal is to discuss what we want the framework to look like and start the process which will require separate discussions around criteria, an initial set of collections, and concrete policy updates.
Anyone’s feedback on this proposal is much appreciated.
Then we could start a vote on moving toward it or keeping the approach as-is.
If this initiative is supported, I’ll volunteer to lead / contribute actively to the policy change effort.
Sounds like a good idea to me. I think we should have one/two places where:
collection maintainers can apply for inclusion by trying to convince us why their collection would be a good fit for the Ansible package;
users can request / vote for collections to be considered for inclusion.
It might also make sense to have a subgroup of the Steering Committee doing the selection, and the whole group only being involved in edge cases / when the subgroup cannot decide on something.
And another thought as a separate comment: we should handle (relatively) small community collections differently then non-community collections and large community collections. Otherwise, we would be unfair to folks who do the extra steps of creating a community collection for content, versus just trying to get their stuff into community.general (which will likely end up being included, at least unless we figure out a way to split a “somehow more commonly useful” part of c.g out into a new “community.core” collection).
I think this is a good idea. I do not feel like the package’s current inventory is focused on providing new users with a quick and easy way to use Ansible in a wide range of common use cases.
One (maybe unintentional) benefit of the current package/docs that I like a lot is being able to discover other collections. I have been on the docs page looking for a specific thing and accidentally found collections that I find interesting or potentially useful.
I am not sure how that discovery could be maintained (maybe a quarterly spotlight post or page on the docs site?) but maybe something worth considering as you move forward
edit: I know you can search in galaxy, but new users probably dont use galaxy UI much. I know I did not really use galaxy UI until recently, when I had to start uploading/maintaining collections
This has come up repeatedly over the last six years; there also have been several inclusion requests that were mainly about getting collections on docs.ansible.com… I think the idea was to let Galaxy take over that part of showing collection documentation (especially versioned!) and making discovery easy.
Unfortunately right now (as for several years now), Galaxy’s docs viewer is not doing so great (like docs for some plugins isn’t shown because they use doc fragments from other collections; linking to other plugins/modules doesn’t work right now; browsing documentation for larger collections is not very convenient right now; …), it’s rather slow (I don’t mean the recent extreme slowliness), and discovery of content where you don’t know the collecftion name yet is still not so great.
Good to know others have found it useful! I think 90% of the value I’ve gotten from discovery is just knowing a collection exists. In other words, reproducing the full docs subtree for a collection is probably overkill. Just a quick blurb (README.md snippet) and a list of modules would be enough for me to decide if I need to look deeper
I not sure what you mean by this. Does “trim down” also include removing collections? I tend to not do this and would keep the collections that are already part of the community package.
But I generally agree with the rest, we should be way more restrictive when it comes to collection inclusion. Although it will be difficult (not impossible!) to define the criteria when inclusion of a collection is OK. But that’s off-topic here, as far as I understand.
It is a LOT harder to remove something people already have than to restrict new things,we have learnt this in core the HARD way.
One ‘solution’ would be to have the docs site reference galaxy for plugin documentation instead of presenting it it self, thought I would consider this a non starter with the current state of galaxy docs.
Even w/o that change, making galaxy a better way to discover and document collections would help alleviate the pressure for inclusion into the community package. Sadly I don’t think there are resources to make that happen right now.
Also, galaxy being a very dynamic and data driven site, it is expected that performance and problems crop up a lot more than for a static site like docs is.
Thanks everyone for your comment!
How about the following long term plan (taking trimming down and backwards compatibility into consideration), i.e. a sequence of topics to eat the elephant one bite at a time:
In this proposal: We vote for/against stopping including new collections
Next topic (if topic 1 resulted in “for stopping”): We discuss and vote on specific criteria for inclusion. All will get reflected in the current inclusion policy doc on docs.ansible.com
Next topic (if previous topic resulted in “for”): We discuss trimming down the package to core and a limited set of collections. We’ll use the criteria we’ve developed in the previous topic. We’ll vote on it. If not supported, we will not proceed further.
Next topic (if trimming down in topic 3 is supported): we’ll discuss community.general collection split, vote.
Next topic (if trimming down in topic 3 is supported): We’ll discuss and vote on a deprecation plan for all collections we haven’t decided to leave in the package.
I would suggest amending the proposal from ‘stopping’ to ‘creating a much higher bar’ for inclusion, just to allow for some collection that provides exceptional value to still be added.
I cannot think of one right now, but if being a OSS maintainer has taught me anything, people can be very creative when subverting your expectations.
@russoz sure, it’s not, it’s a potential future topic
I would suggest amending the proposal from ‘stopping’ to ‘creating a much higher bar’ for inclusion, just to allow for some collection that provides exceptional value to still be added.
@bcoca sure, that’s the plan, for the topic from step 2