# \[VOTE ENDED 2026-03-26\] Collection inclusion requirements update proposal: mandate testing against devel or milestone branch of ansible-core **URL:** \ **Category:** Project Discussions\ **Tags:** resolved, community-wg, collections, network\ **Created:** [March 5, 2026, 7:37am UTC](https://forum.ansible.com/t/vote-ended-2026-03-26-collection-inclusion-requirements-update-proposal-mandate-testing-against-devel-or-milestone-branch-of-ansible-core/45486 "2026-03-05T07:37:43Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![Andersson007](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/andersson007/32/89_2.png) [@Andersson007](https://forum.ansible.com/u/Andersson007)\ **Post date:** [March 5, 2026, 7:37am UTC](https://forum.ansible.com/t/vote-ended-2026-03-26-collection-inclusion-requirements-update-proposal-mandate-testing-against-devel-or-milestone-branch-of-ansible-core/45486/1 "2026-03-05T07:37:43Z")
Hello Steering Committee, During the Ansible Contributor Summit at CfgMgmtCamp, we (including SC members) discussed improving the stability of the community package. Current [collection requirements](https://docs.ansible.com/projects/ansible/devel/community/collection_contributors/collection_requirements.html#ci-testing) only mandate testing against stable `ansible-core` versions. While many maintainers voluntarily test against `devel`, failing to catch regressions early leads to: - **User friction:** Problems are only discovered after a core release. - **Cascading failures:** Broken collections often break their dependents, impacting the wider ecosystem. To ensure content stability, I propose requiring collections to run tests against the `devel` or `milestone` branches of `ansible-core`. **Proposed changes:** [PR #3511](https://github.com/ansible/ansible-documentation/pull/3511) I look forward to hearing the Committee’s thoughts on this requirement. ---
**Author:** ![kks](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/kks/32/10352_2.png) [@kks](https://forum.ansible.com/u/kks)\ **Post date:** [March 5, 2026, 8:55am UTC](https://forum.ansible.com/t/vote-ended-2026-03-26-collection-inclusion-requirements-update-proposal-mandate-testing-against-devel-or-milestone-branch-of-ansible-core/45486/2 "2026-03-05T08:55:31Z")
I am not from any commitee, but if you mandate to test agaist devel you have to provide tools to isolate tests. As testing agains moving branch will just lead to flaky tests. Let me explain. If I run tests against devel every day - even if I do not change anything in my collection / content tests might fail just because devel has an issue or just in a middle of refactoring. But it I test agains specific commit in devel - every time tests run if collection has not been changed result will be the same (devel frozen to specific commit). Question is how to keep this commit I am running againts moving together with devel. That might be some tool (or its extension) like dependabot or renovate. Renovate seems to have an option to follow git references - [Git Refs - Renovate Docs](https://docs.renovatebot.com/modules/datasource/git-refs/) . This update to git reference (commit) to devel should run every week or month (depends on maintainer and how ofter they relese) and create PRs. This will allow to fix all test depedencies. So when there will be failure because of devel update it will be evident what change (from what ref to what ref caused it). ---
**Author:** ![dbrennand](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/dbrennand/32/7830_2.png) [@dbrennand](https://forum.ansible.com/u/dbrennand)\ **Post date:** [March 5, 2026, 9:11am UTC](https://forum.ansible.com/t/vote-ended-2026-03-26-collection-inclusion-requirements-update-proposal-mandate-testing-against-devel-or-milestone-branch-of-ansible-core/45486/3 "2026-03-05T09:11:37Z")
IMO I think this requirement would be good for the wider ecosystem and allow maintainers to catch changes in `ansible-core` early. If this kind of requirement was in place before the `ansible-core` 2.19 release then _possibly_ the whole saga with `ansible.netcommon` may have been caught earlier as the CI would have been failing. > [@kks](#): > > Let me explain. If I run tests against devel every day - even if I do not change anything in my collection / content tests might fail just because devel has an issue or just in a middle of refactoring. @kks makes a **very** good point here. The `community.beszel` collection has hit this exact issue. We’re testing against `devel` and the CI is broken right now because of changes to the `user` module, so one of the roles is failing in the integration tests: [ci(fix): Resolve CI issue with append and groups being missing · Issue #36 · ansible-collections/community.beszel · GitHub](https://github.com/ansible-collections/community.beszel/issues/36). I haven’t had time to fix the CI yet. I was just reading through: [ansible-core project branches and tags — Ansible Community Documentation](https://docs.ansible.com/projects/ansible/latest/dev_guide/core_branches_and_tags.html) and it seems like the `milestone` branch may be better for this than `devel`. > A `milestone` branch is a slow-moving stream of the `devel` branch, intended for external testing of `ansible-core` features under active development. ---
**Author:** ![Andersson007](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/andersson007/32/89_2.png) [@Andersson007](https://forum.ansible.com/u/Andersson007)\ **Post date:** [March 5, 2026, 9:26am UTC](https://forum.ansible.com/t/vote-ended-2026-03-26-collection-inclusion-requirements-update-proposal-mandate-testing-against-devel-or-milestone-branch-of-ansible-core/45486/4 "2026-03-05T09:26:37Z")
> [@kks](#): > > As testing agains moving branch will just lead to flaky tests. to me (a maintainer of a few collections) it doesn’t feel like a big problem: 1. doesn’t happen too often 2. you can ignore failures in PRs until you fix them - yes, it doesn’t look very good, but one failure from typically tens of others green checks we run in collections doesn’t look too scary It’s my personal view ^, though I see your concern and situations can be different ! Anyway, for people who wants more stability, there’s an option to test against the `milestone` branch instead of `devel` as PR suggests ---
**Author:** ![sivel](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/sivel/32/1198_2.png) [@sivel](https://forum.ansible.com/u/sivel)\ **Post date:** [March 5, 2026, 3:01pm UTC](https://forum.ansible.com/t/vote-ended-2026-03-26-collection-inclusion-requirements-update-proposal-mandate-testing-against-devel-or-milestone-branch-of-ansible-core/45486/5 "2026-03-05T15:01:54Z")
We’ve generally said that if you test against devel, it should be “non voting”, meaning it should not impact normal operations if it causes issues. Depending on the setup, a nightly or weekly job could still report failures against devel. The milestone branch was created to give an alternative that was closer to devel, but had predictable updates and was consistent between those updates. As of the 2.21 release schedule, we’ve switched from 3 dev phases (when the milestone branch was advanced) to 5 or 6 dev phases, and as a result the advancements are more frequent (4 weeks instead of 8). ---
**Author:** ![felixfontein](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/felixfontein/32/7530_2.png) [@felixfontein](https://forum.ansible.com/u/felixfontein)\ **Post date:** [March 5, 2026, 8:15pm UTC](https://forum.ansible.com/t/vote-ended-2026-03-26-collection-inclusion-requirements-update-proposal-mandate-testing-against-devel-or-milestone-branch-of-ansible-core/45486/6 "2026-03-05T20:15:23Z")
I personally prefer `devel` over `milestone`, for mainly two reasons: - `milestone` changes on certain dates, and then you get a month worth of changes. If you can plan to invest enough time in your collection every time `milestone` is bumped, this should be fine. But if you can’t plan that much time in ahead of time for these fixed dates, this isn’t great. - It can be that something in `milestone` is broken. This won’t get fixed until `milestone` is updated again. (Bugs in `devel` might already get fixed a few days or even just hours later.) Also you won’t be able to test bigger changes when they get merged, only some weeks later. For example, there are some bigger PRs open in ansible/ansible right now (projections + setting vars, support for old Python versions, …) that haven’t been merged yet or were merged after the start of the last development cycle (March 2nd), so you will only encounter them in CI when `milestone` is bumped again - basically when ansible-core 2.21.0b1 is released (April 6th). This reduces the time you have to react to these big changes, and potentially find bugs in these changes that affect your code (and getting them fixed before the release candidates start). As opposed to the large changes of ansible-core 2.19 the changes that are waiting to get merged now likely don’t affect many collections, so this likely isn’t a problem, but for some this might be. I want to emphasize again that the above points are _my_ reasons to prefer `devel` over `milestone`, and that for other people these points could be less relevant or totally irrelevant. For that reason requiring that at least one of `devel` and `milestone` is used sounds good to me. The scenario @kks describes is a perfect mix of `devel` and `milestone` IMO, since the maintainers control when the branch is advanced, and first run CI to see whether it breaks something before bumping `devel`. This might be the best solution for most maintainers for whom simply using `devel` is too much work / too dangerous, and who also share the concers I have for `milestone`. (You could even set up something so the update PRs are automatically merge if CI passes. That way you only have to take a look when something fails, and then you don’t have that much pressure as with `devel` or `milestone` to fix it.) We definitely have to make sure that our formulation of the rule allows to proceed this way as well. (I.e. fix a commit on `devel` and update it at least ~once per month to the latest.) ---
**Author:** ![mariolenz](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/mariolenz/32/108_2.png) [@mariolenz](https://forum.ansible.com/u/mariolenz)\ **Post date:** [March 8, 2026, 7:03pm UTC](https://forum.ansible.com/t/vote-ended-2026-03-26-collection-inclusion-requirements-update-proposal-mandate-testing-against-devel-or-milestone-branch-of-ansible-core/45486/7 "2026-03-08T19:03:04Z")
I basically agree that this is a good requirement. But since this is “just” about finding out about new linting requirements early, should we mention that it’s OK if those tests are non-voting? Although there’s always the risk that people ignore failures when the tests are non-voting. Should we also explain the pros and cons of testing with devel and milestone here, or somewhere else and link to this? Or does there already exist such an explanation somewhere that we could link to? All in all, sounds like a good idea to me. Let’s just clarify some details and the small print. ---
**Author:** ![felixfontein](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/felixfontein/32/7530_2.png) [@felixfontein](https://forum.ansible.com/u/felixfontein)\ **Post date:** [March 8, 2026, 8:42pm UTC](https://forum.ansible.com/t/vote-ended-2026-03-26-collection-inclusion-requirements-update-proposal-mandate-testing-against-devel-or-milestone-branch-of-ansible-core/45486/8 "2026-03-08T20:42:58Z")
> [@mariolenz](#): > > But since this is “just” about finding out about new linting requirements early, That’s not true; it’s also about integration or unit tests failing early because of changes in ansible-core that affect something used by the collection. > [@mariolenz](#): > > should we mention that it’s OK if those tests are non-voting? Although there’s always the risk that people ignore failures when the tests are non-voting. That’s a good question. I am tempted to say that non-voting is fine, but I’m a bit worried that it’s taken as an excuse to simply ignore the results, which is contrary to what we want to achieve by requiring to test against these branches. On the other hand, it’s better to have some “always red” as a remainder that something isn’t fully right. ---
**Author:** ![felixfontein](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/felixfontein/32/7530_2.png) [@felixfontein](https://forum.ansible.com/u/felixfontein)\ **Post date:** [March 9, 2026, 5:56am UTC](https://forum.ansible.com/t/vote-ended-2026-03-26-collection-inclusion-requirements-update-proposal-mandate-testing-against-devel-or-milestone-branch-of-ansible-core/45486/9 "2026-03-09T05:56:44Z")
BTW, the `milestone` branch was bumped yesterday, not on March 2nd as announced in [Ansible-core 2.21 — Ansible Community Documentation](https://docs.ansible.com/projects/ansible/devel/roadmap/ROADMAP_2_21.html#development-phase) (I know because I have a CI job in antsibull-nox which tells me when this happens, so I can adjust `milestone`’s supported Python versions in case they change.) ---
**Author:** ![mariolenz](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/mariolenz/32/108_2.png) [@mariolenz](https://forum.ansible.com/u/mariolenz)\ **Post date:** [March 9, 2026, 4:11pm UTC](https://forum.ansible.com/t/vote-ended-2026-03-26-collection-inclusion-requirements-update-proposal-mandate-testing-against-devel-or-milestone-branch-of-ansible-core/45486/10 "2026-03-09T16:11:25Z")
> [@felixfontein](#): > > > [@mariolenz](#): > > > > But since this is “just” about finding out about new linting requirements early, > > That’s not true; it’s also about integration or unit tests failing early because of changes in ansible-core that affect something used by the collection. I agree, but the PR currently says: > You MUST run the `ansible-test sanity` command against the `devel` or `milestone` branch of `ansible-core` so that you find out about new linting requirements earlier. That’s why I said it’s just about finding out about new linting requirements. Maybe we should drop the “linting”, then, and only mention “requirements”? Or maybe change this to > so that you find out about issues with the next `ansible-core` release early or something similar. ---
**Author:** ![felixfontein](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/felixfontein/32/7530_2.png) [@felixfontein](https://forum.ansible.com/u/felixfontein)\ **Post date:** [March 9, 2026, 5:37pm UTC](https://forum.ansible.com/t/vote-ended-2026-03-26-collection-inclusion-requirements-update-proposal-mandate-testing-against-devel-or-milestone-branch-of-ansible-core/45486/11 "2026-03-09T17:37:09Z")
Sorry, I didn’t had time to look at the PR yet. I was assuming that this is about generally testing the collection, and not just about sanity tests. The PR only requires that sanity tests are run with ansible-core devel or milestone, and actually drops the requirement that all of CI runs with all supported ansible-core versions (I wrote a comment on that). I think we should rather require that all of CI is also run with devel or milestone (or “something inbetween”, i.e. what @kks sugested). ---
**Author:** ![Andersson007](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/andersson007/32/89_2.png) [@Andersson007](https://forum.ansible.com/u/Andersson007)\ **Post date:** [March 10, 2026, 8:56am UTC](https://forum.ansible.com/t/vote-ended-2026-03-26-collection-inclusion-requirements-update-proposal-mandate-testing-against-devel-or-milestone-branch-of-ansible-core/45486/12 "2026-03-10T08:56:10Z")
Yeah, it’s not only about the linting requirements, that wording was there before. I’ve updated the wording [[SC vote is needed] collection\_requirements.rst: add a requirement to test against devel or milestone by Andersson007 · Pull Request #3511 · ansible/ansible-documentation · GitHub](https://github.com/ansible/ansible-documentation/pull/3511/changes/7eb095121d116fc5c12d79365734a9c3cdcb6846) PTAL ---
**Author:** ![Andersson007](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/andersson007/32/89_2.png) [@Andersson007](https://forum.ansible.com/u/Andersson007)\ **Post date:** [March 11, 2026, 10:30am UTC](https://forum.ansible.com/t/vote-ended-2026-03-26-collection-inclusion-requirements-update-proposal-mandate-testing-against-devel-or-milestone-branch-of-ansible-core/45486/13 "2026-03-11T10:30:50Z")
Please folks take a look at the latest wording [[SC vote is needed] collection\_requirements.rst: add a requirement to test against devel or milestone by Andersson007 · Pull Request #3511 · ansible/ansible-documentation · GitHub](https://github.com/ansible/ansible-documentation/pull/3511) I’d like to start a poll next week if no objections ---
**Author:** ![mariolenz](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/mariolenz/32/108_2.png) [@mariolenz](https://forum.ansible.com/u/mariolenz)\ **Post date:** [March 11, 2026, 5:24pm UTC](https://forum.ansible.com/t/vote-ended-2026-03-26-collection-inclusion-requirements-update-proposal-mandate-testing-against-devel-or-milestone-branch-of-ansible-core/45486/14 "2026-03-11T17:24:19Z")
At first glance, this looks good to me. I’ve just found a (probably very minor) wording issue. And I’m not even sure there, because I’m not a native speaker of English. I’ve added a comment about this. ---
**Author:** ![dbrennand](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/dbrennand/32/7830_2.png) [@dbrennand](https://forum.ansible.com/u/dbrennand)\ **Post date:** [March 12, 2026, 12:05pm UTC](https://forum.ansible.com/t/vote-ended-2026-03-26-collection-inclusion-requirements-update-proposal-mandate-testing-against-devel-or-milestone-branch-of-ansible-core/45486/15 "2026-03-12T12:05:38Z")
When is it expected for collections to comply with this change? There will be a certain period of time for collections to update their CI for this new requirement? How long would that be? Is this already documented somewhere generally for new inclusion requirements? ---
**Author:** ![mariolenz](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/mariolenz/32/108_2.png) [@mariolenz](https://forum.ansible.com/u/mariolenz)\ **Post date:** [March 12, 2026, 7:34pm UTC](https://forum.ansible.com/t/vote-ended-2026-03-26-collection-inclusion-requirements-update-proposal-mandate-testing-against-devel-or-milestone-branch-of-ansible-core/45486/16 "2026-03-12T19:34:22Z")
Just to have an example that we can talk about, would the [workflows in community.vmware](https://github.com/ansible-collections/community.vmware/tree/main/.github/workflows) be OK? Disclaimer: I have a personal interest that they are 😏 I think it would be easier for me to understand what this means if I could see an example. And / or to see a possible implementation of this in the [collection\_template](https://github.com/ansible-collections/collection_template). Or is the collection template already OK and we just have to make things clearer in the documentation? Otherwise, I think we also have to update the template. ---
**Author:** ![dbrennand](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/dbrennand/32/7830_2.png) [@dbrennand](https://forum.ansible.com/u/dbrennand)\ **Post date:** [March 13, 2026, 9:30am UTC](https://forum.ansible.com/t/vote-ended-2026-03-26-collection-inclusion-requirements-update-proposal-mandate-testing-against-devel-or-milestone-branch-of-ansible-core/45486/17 "2026-03-13T09:30:45Z")
@mariolenz Yes, I think it’s a good example to discuss 👍 It looks like the collection’s CI is already compliant with this requirement as `milestone` is being tested in [sanity](https://github.com/ansible-collections/community.vmware/blob/main/.github/workflows/ansible-test.yml#L87) and [unit](https://github.com/ansible-collections/community.vmware/blob/main/.github/workflows/ansible-test.yml#L138) tests. It’s also running once a day which is far more frequent than the once per week requirement. EDIT: As for the collection template, the workflow is already set up to test against `devel` and runs daily: - [collection\_template/.github/workflows/ansible-test.yml at main · ansible-collections/collection\_template · GitHub](https://github.com/ansible-collections/collection_template/blob/main/.github/workflows/ansible-test.yml#L82) - [collection\_template/.github/workflows/ansible-test.yml at main · ansible-collections/collection\_template · GitHub](https://github.com/ansible-collections/collection_template/blob/main/.github/workflows/ansible-test.yml#L132) ---
**Author:** ![Andersson007](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/andersson007/32/89_2.png) [@Andersson007](https://forum.ansible.com/u/Andersson007)\ **Post date:** [March 16, 2026, 7:31am UTC](https://forum.ansible.com/t/vote-ended-2026-03-26-collection-inclusion-requirements-update-proposal-mandate-testing-against-devel-or-milestone-branch-of-ansible-core/45486/18 "2026-03-16T07:31:16Z")
@mariolenz collection\_template is already OK as I see ---
**Author:** ![Andersson007](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/andersson007/32/89_2.png) [@Andersson007](https://forum.ansible.com/u/Andersson007)\ **Post date:** [March 19, 2026, 7:05am UTC](https://forum.ansible.com/t/vote-ended-2026-03-26-collection-inclusion-requirements-update-proposal-mandate-testing-against-devel-or-milestone-branch-of-ansible-core/45486/19 "2026-03-19T07:05:07Z")
[@SteeringCommittee](https://forum.ansible.com/groups/steeringcommittee) and community, please vote. The poll will end (if no objections) 2026-03-26. If the proposal is accepted: - We’ll announce the change via Bullhorn and under `news-for-maintainers` tag - New inclusion candidates will have to abide; the existing collections will have at least a couple of months. Anyway, Mario always creates an issue in problematic collections first before initiating removal, so maintainers will have plenty of time to adjust. Please vote. _Poll ([view on site](https://forum.ansible.com/t/vote-ended-2026-03-26-collection-inclusion-requirements-update-proposal-mandate-testing-against-devel-or-milestone-branch-of-ansible-core/45486/19))_ _Poll ([view on site](https://forum.ansible.com/t/vote-ended-2026-03-26-collection-inclusion-requirements-update-proposal-mandate-testing-against-devel-or-milestone-branch-of-ansible-core/45486/19))_ ---
**Author:** ![gundalow](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/gundalow/32/99_2.png) [@gundalow](https://forum.ansible.com/u/gundalow)\ **Post date:** [March 19, 2026, 10:23am UTC](https://forum.ansible.com/t/vote-ended-2026-03-26-collection-inclusion-requirements-update-proposal-mandate-testing-against-devel-or-milestone-branch-of-ansible-core/45486/20 "2026-03-19T10:23:40Z")
Assuming the [proposal is merged](https://github.com/ansible/ansible-documentation/pull/3511/) , let’s ensure that the GitHub Actions and in-line documentation in those workflows is updated. Even for non-included collections, we can still guide people in the right direction. Having a weekly scheduled job against `devel` would help all collection maintainers. [Next page](https://forum.ansible.com/t/vote-ended-2026-03-26-collection-inclusion-requirements-update-proposal-mandate-testing-against-devel-or-milestone-branch-of-ansible-core/45486.md?page=2)