Future of the Ansible Community package

[Denney-tech (Caleb Denney)]

I would vote very much in favor of option 4.

[briantist]

I have a hard time understanding option 2 (and by extension option 4). It feels like still having a package with included collections, and then adding a separate process and criteria for “approved” collections (and their inclusion into docs) is only adding complication (and frankly, confusion).

@acozine

I see several posts that say, roughly, “Reason X is not a good reason to keep the Ansible package.” I see much less about the reasons why people think it would be good to get rid of the Ansible package. What problem are we trying to solve?

Another way to look at it is this. How would folks fill in one or more blanks:

• “If we stopped maintaining the batteries-included Ansible package, we could stop ______.”
• “If we stopped maintaining the batteries-included Ansible package, we could start _____.”
• “If we stopped maintaining the batteries-included Ansible package, we would _________,”

^ this. I’m still unclear what problem we’re really trying to solve and how that split works toward that.

@Leo

IMO a lot of users start with the community package. It’s a great entry point, and also a great way to foster engagement within the community, particularly for those who strive to be included and see recognition in that.

I also see the Ansible Community Package as a way to advocate and enable people into recommended practices for Ansible collection development.

I agree, but I think we also need to accept (and embrace!) that for many users, the package is what they want and need, and they will never switch to using individual collections. We are all so heavily on the developer/maintainer side of things that I think it’s really easy to fall into the trap of forgetting that our user experience just isn’t the same as most of ansible’s user base.

@markuman

Basically an influx of new inclusion requests is the best what can happen for ansible. That just means that ansible live. People are using it and want to use it more.

This also resonated with me.

[Leo (Leo Gallego)]

@briantist

I have a hard time understanding option 2 (and by extension option 4). It feels like still having a package with included collections, and then adding a separate process and criteria for “approved” collections (and their inclusion into docs) is only adding complication (and frankly, confusion).

In case of option 2, my understanding is that the collections are pretty much static, there is a very small subset and although that might change with some process, it looks to be a very minimal bundle. Basically moves all the recognition/validation process to the badge instead of the inclusion itself, but the work related to the “inclusion request” would be mostly the same (in my point of view), while losing the benefit for those looking for the “batteries included” package.

In case of option 4, the “separate process” can be an additional checkbox at the end. As in: you need to pass all the “recommended practices/approved rules” + some “inclusion rule” to make it into the package. This way, we get a validation for all those that care about it, and an additional step, which can have stricter maintenance requirements, for package inclusion.

@briantist

I agree, but I think we also need to accept (and embrace!) that for many users, the package is what they want and need, and they will never switch to using individual collections. We are all so heavily on the developer/maintainer side of things that I think it’s really easy to fall into the trap of forgetting that our user experience just isn’t the same as most of ansible’s user base.

I agree, and that was my reason to propose something less drastic. I don’t see the need to stop maintaining the batteries-included package, but it might need to evolve to make maintenance and inclusion more sustainable.

[samccann (Sandra McCann)]

@briantist

I agree, but I think we also need to accept (and embrace!) that for many users, the package is what they want and need, and they will never switch to using individual collections. We are all so heavily on the developer/maintainer side of things that I think it’s really easy to fall into the trap of forgetting that our user experience just isn’t the same as most of ansible’s user base.

This is my biggest fear, that we are in our own little bubble here, and if we do change the package dramatically, we won’t hear the screams until it’s too late. I know we’ve tried publicizing this in every way we can think of, but I feel the silent user base not participating will roast us over the coals if we remove significant collections from the package.

I know this was just a straw poll to see which way the wind’s blowing, but if we are serious about big changes, I think we get @gwmngilfen help to create an official survey and pop it up on the docs banners. Community surveys get I think over 100 responses in the past, which is still a minor fraction of our user base, but it would get attention

[felixfontein]

While we’re at it, I have another option: have (at least) two ansible community packages, one with a more limited set of collections, and one larger, that includes a lot more. This would be similar for example to how texlive is packaged in Debian: there’s texlive-base, a minimal system (similar to ansible-core); there’s texlive-full, which includes basically everything you might need (or not), and there’s texlive, a “decent selection of the TeX Live packages”, which serves as a middle-ground that’s fine for many users who don’t want to install invidiual packages, but also don’t want the full thing (which is multiple gigabytes IIRC).

So basically there are three (community) tiers of collections: a ‘core’ set of essential collections (included in the smaller package), an ‘extended’ set of collections that are approved (similar to the Galaxy badge mentioned above), and everything else that’s on Galaxy. But instead of simply having a badge to make it easier for users to identify these collections, there’s also an official package containing them.

(I’m fully aware that this is more work, while the current iteration of this discussion originated from another discussion about reducing the amount of work due to limited resources )

[briantist]

as you said, it’s more work, so I would come back to the question of what we (or the community) gains from this. The current package is nowhere close to multiple GB, and although people keep talking about the size of the package and fears of “bloat” I still haven’t heard a good reason why the current size is a problem, nor any reason why we think the size would grow to problematic proportions.

@samccann

we won’t hear the screams until it’s too late

I also worry about this.

@samccann

if we are serious about big changes, I think we get @gwmngilfen help to create an official survey and pop it up on the docs banners

Yeah I think you’re right and if this is the way the maintainers want to go we ought to make a larger effort to pull in that silent majority and get what feedback we can.

[samccann (Sandra McCann)]

@acozine

Another way to look at it is this. How would folks fill in one or more blanks:

I’m going to take a stab at this:

• “If we stopped maintaining the batteries-included Ansible package, we could stop the inclusion reviews, the monthly minor releases, the major releases and the community package documentation.”
• “If we stopped maintaining the batteries-included Ansible package, we could start focusing on improving important existing collections, enhancing community around other ansible ecosystem projects such event-driven Ansible.”
• “If we stopped maintaining the batteries-included Ansible package, we would still need a solution for those users who cannot use galaxy for $REASONS.

Personally, that last one is the most important in my mind. I feel like there are many people who have valid reasons for not using or not being able to use galaxy, and we’d need to continue to have a solution for those users, or we’ll dramatically hurt the Ansible community.

[utoddl (Todd Lewis)]

Maybe a q. for @gwmngilfen as it’s a metrics sort of thing, but how much churn occurs in the batteries-included collections as a result of simply staying forward-compatible? By that I mean in contrast to people actively tweaking, new features, etc., but rather just keeping existing automated tests and linting happy as ansible-core grows up?

Related questions:

• Who picks up that work now?
• If the big blob of collections gets somehow de-blobed, then who would pick up that work?

[sivel]

It should also note that a downside that I’ve seen to the ever growing package is that people incorrectly infer support and quality with the inclusion in the ansible package.

Unfortunately, neither of those are realistically true. Some collections and content are well supported and well thought out, but some aren’t. At this point, I’ve largely stopped using any module external to ansible-core. Every now and then I may vendor a small subset of functionality if it’s a toy project, or more likely just write my own module/plugin.

I’ve experienced quite a number of modules that simply don’t do their job well, or shouldn’t have been written to begin with. I’ve heard this same experience from a number of users.

So while you are getting a lot of content, there is only so much that the review process and sanity tests can ensure. It’s always been a problem that for a certain quanitity of plugin authors that the inclusion is more marketing than actually filling a useful gap.

Where has the 80/20 rule gone? Will 80% of the users find this useful, and need it?

[gotmax23 (Maxwell G (He/They))]

@markuman

Stop adding new packages is not an option, because there are no valid arguments for it.

I think there are . There are already over 100 collections in the package. The build process is lengthy. Building The files take up a (relatively) large amount of disk space, especially if you’re including it in a container image. I do not want the package to balloon to such an extent that it’s no longer useful. Maybe we do not want to completely stop reviewing collections, but I think we should at least have a more narrow view of what types of collections to include. For example, I am not convinced that we should keep including new vendor collections when their customers could easily download the collection from Ansible Galaxy/AAH. These type of collections are not widely useful in the way that ansible.posix or community.general are.

@Andersson007

Option 2: Stop including new collections, trim down the package to ansible-core + a few collections

I do think the current package is too large, but completely changing the contents of ansible and calling it the same thing does not seem like a workable solution.

@felixfontein

While we’re at it, I have another option: have (at least) two ansible community packages, one with a more limited set of collections, and one larger, that includes a lot more.

I’m open to the idea, but I think our main focus should first be completely automating the current process so releasing another package isn’t so onerous.

@sivel

Unfortunately, neither of those are realistically true. Some collections and content are well supported and well thought out, but some aren’t. At this point, I’ve largely stopped using any module external to ansible-core. Every now and then I may vendor a small subset of functionality if it’s a toy project, or more likely just write my own module/plugin.

I wouldn’t go that far, but yeah, there are definitely certain collections in the package with quality issues.

@sivel

Where has the 80/20 rule gone? Will 80% of the users find this useful, and need it?

Definitely.

[Andersson007]

@briantist

I have a hard time understanding option 2 (and by extension option 4). It feels like still having a package with included collections, and then adding a separate process and criteria for “approved” collections (and their inclusion into docs) is only adding complication (and frankly, confusion

yes, i agree, i wouldn’t be in favor of any parallel requirements and processes as unnecessary complications.

@felixfontein

I’m fully aware that this is more work, while the current iteration of this discussion originated from another discussion about reducing the amount of work due to limited resources

hehe, yep, would be nice to see as a result of the discussion some simplification, not complications imo

@samccann

if we are serious about big changes, I think we get @gwmngilfen help to create an official survey and pop it up on the docs banners.

SGTM

@sivel

It should also note that a downside that I’ve seen to the ever growing package is that people incorrectly infer support and quality with the inclusion in the ansible package.

Unfortunately, neither of those are realistically true. Some collections and content are well supported and well thought out, but some aren’t.

Agreed

@gotmax23

I do think the current package is too large, but completely changing the contents of ansible and calling it the same thing does not seem like a workable solution.

Good point

[markuman (Markus)]

@gotmax23

@markuman

Stop adding new packages is not an option, because there are no valid arguments for it.

I think there are . There are already over 100 collections in the package. The build process is lengthy. Building The files take up a (relatively) large amount of disk space, especially if you’re including it in a container image. I do not want the package to balloon to such an extent that it’s no longer useful. Maybe we do not want to completely stop reviewing collections, but I think we should at least have a more narrow view of what types of collections to include. For example, I am not convinced that we should keep including new vendor collections when their customers could easily download the collection from Ansible Galaxy/AAH. These type of collections are not widely useful in the way that ansible.posix or community.general are.

Ok, got it.
So stopping new packages is not an option depends on the result in my point of view.

@felixfontein

While we’re at it, I have another option: have (at least) two ansible community packages, one with a more limited set of collections, and one larger, that includes a lot more. This would be similar for example to how texlive is packaged in Debian: there’s texlive-base, a minimal system (similar to ansible-core); there’s texlive-full, which includes basically everything you might need (or not), and there’s texlive, a “decent selection of the TeX Live packages”, which serves as a middle-ground that’s fine for many users who don’t want to install invidiual packages, but also don’t want the full thing (which is multiple gigabytes IIRC).

That sounds like a good compromise to me.
Current ansible would become ansible-full and just ansible will be reduced somehow, like the 80/20 rule that @sivel mentioned.

[bcoca (Brian Coca)]

If the ‘multiple packages’ is the route most want to continue along would drop the current ansible package and create carefully named packages:

• ansible-community-core/quality/starter/simple/… (ansible-* + choice collections most people would need)
• ansible-kitchensync/all/diy/… (name says all)

I would suggest some way to indicate good support/tests/quality

• ansible-com-gold (all ansible-* + choice well maintained collections)
  …
• ansible-com-tin (this one has community.general)

[samccann (Sandra McCann)]

We also have I think community-minimal EE (just core?) and community-base EE, so would be good to follow a similar naming…if we go that route.

[Denney-tech (Caleb Denney)]

@bcoca In the multi-package route, I think the ansible package could remain the same all-inclusive tier that it has always been, and make the new alternate packages be subsets of packages like you suggest.

[bcoca (Brian Coca)]

@bcoca In the multi-package route, I think the ansible package could remain the same all-inclusive tier that it has always been, and make the new alternate packages be subsets of packages like you suggest.

The problem with that is the current state, which is becoming unmaintainable, will be kept, the point of removing the ansible name is making sure users know there is a departure. Not doing this was one of the biggest sore points in 2.9/2.10 transition.

[Denney-tech (Caleb Denney)]

How would the ansible-kitchensync be any different? Or are you just trying to disassociate users from the “expectation of quality?”

[bcoca (Brian Coca)]

“eggzactly” fill20chars

[Denney-tech (Caleb Denney)]

Fair enough. mustb20

[bcoca (Brian Coca)]

One way kitchensync can be made ‘simpler’ is to just include everything under the community namespace in github and/or badge in galaxy, no tests, no build time, jsut ‘gimme all on interwebs’ approach. This still gives the user a ‘full package’ but offloads most of the maintenance from the ‘engaged community’.