# Unified Collection Testing Strategy - Kick off & Landscape Overview **URL:** \ **Category:** Collection Development\ **Tags:** azure, fedora, ubuntu, collection-test-strat, ansible-lint, galaxy-ng\ **Created:** [May 14, 2026, 4:23pm UTC](https://forum.ansible.com/t/unified-collection-testing-strategy-kick-off-landscape-overview/45799 "2026-05-14T16:23:45Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![dbrennand](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/dbrennand/32/7830_2.png) [@dbrennand](https://forum.ansible.com/u/dbrennand)\ **Post date:** [May 14, 2026, 4:23pm UTC](https://forum.ansible.com/t/unified-collection-testing-strategy-kick-off-landscape-overview/45799/1 "2026-05-14T16:23:45Z")
Hey Ansible Community, As Don shared in his [Ansible community roadmap 2026](https://forum.ansible.com/t/ansible-community-roadmap-2026/45522) post, and as mentioned in [Clarifying the Role of ansible-test in Collection Testing](https://forum.ansible.com/t/clarifying-the-role-of-ansible-test-in-collection-testing/45763) a key area for the Community & Partner Engineering team is [Developing a unified strategy for collection testing](https://forum.ansible.com/t/ansible-community-roadmap-2026/45522#p-138470-developing-a-unified-strategy-for-collection-testing-16). In this post I want to kick things off by outlining some of the motivations and goals driving this initiative. I also want to share an overview of the current collection testing landscape to establish a shared understanding for further discussion. ## Why are we doing this? A large motivation for this initiative is direct community feedback from [CfgMgmtCamp 2026](https://forum.ansible.com/t/cfgmgmtcamp-2026-overview/45340) during the contributor summit and other posts such as: - [CfgMgmtCamp 2026 discussion (7/12): Collection testing is still a mess](https://forum.ansible.com/t/cfgmgmtcamp-2026-discussion-7-12-collection-testing-is-still-a-mess/45294) - [Should we be using molecule or ansible-test for unit/integration tests? - Get Help](https://forum.ansible.com/t/should-we-be-using-molecule-or-ansible-test-for-unit-integration-tests/5539) - [Integrating `ansible-test integration` (no pun intended) and `molecule` - Project Discussions](https://forum.ansible.com/t/integrating-ansible-test-integration-no-pun-intended-and-molecule/45018) Another reason we want to create a unified collection testing strategy is to reduce fragmentation across the ecosystem. You might recall Greg’s [Ansible community strategy](https://ansible.github.io/community/posts/ansible_community_strategy_2023.html) from 2023 (and the relevant [XKCD](https://xkcd.com/927/) comic). While Greg’s strategy document focused mainly on fragmentation in the community in areas such as communication and decision making, his analogy about communities of projects being like houses in a neighborhood could also extend to Ansible collections. When you look around the collection ecosystem, you can find most collections use different tools for testing. Going back to Greg’s analogy, it’s fine for projects to do their own thing and make their own decisions inside their own “house” but that can often lead to difficulties across the whole neighborhood. In the case of collections, this often makes it challenging for contributors to work across them. ## What do we want to achieve? With this initiative we want to establish a single, authoritative collection testing strategy that reduces fragmentation caused by multiple overlapping tools both in the Ansible community and in Red Hat. As an outcome of that strategy, we hope to reduce maintenance burden, add consistency for community and Red Hat partner collection maintainers, resulting in content quality improvements across the Ansible collection ecosystem. Some additional benefits that we hope the effort to establish a strategy will achieve: - Identify collection testing areas which need clarity. - Improve documentation of the unified collection testing tooling. - Create migration guides based on the unified collection testing tooling. - Create “AI” skills to assist collection maintainers with the migration to the agreed collection testing tooling. ## Call to Action This initiative to develop a unified collection testing strategy requires participation and collaboration. To get to the best end result, we need to hear as many voices from the community as possible so that we can consider a broad set of views. We ask that you join us in this initiative and help us shape the strategy. As things come into focus and the strategy evolves, we’ll share updates on the forum. We ask that everyone keep an eye on the discussions, get involved, and have your say. For now, here are some ways you can get started in this initiative and participate: - Read through the landscape overview in the below section of this post. Are we overlooking anything? Are there any technical inaccuracies? Do you have any general comments, questions, or suggestions? If so, please reply to this post and let us know. - Review the [pull request in Ansible documentation](https://github.com/ansible/ansible-documentation/pull/3654) that clarifies the role of ansible-test as a collection test tool. - Watch the [collection-test-strat tag](https://forum.ansible.com/tag/collection-test-strat/516) on the forum to follow the discussion. * * * ## Landscape Overview The collection testing landscape as it stands at the time of writing this post has several tools. Let’s start by breaking it down into four areas: unit, integration, sanity and static analysis. Please note that it’s out of scope for this post to explain these areas. There is extensive documentation in the Ansible documentation which you can refer to: - [Testing Ansible - Types of tests](https://docs.ansible.com/projects/ansible/latest/dev_guide/testing.html#types-of-tests) - [Testing Ansible and Collections - Testing an Ansible Collection](https://docs.ansible.com/projects/ansible/latest/dev_guide/testing_running_locally.html#testing-an-ansible-collection) - [Testing Ansible and Collections - Commands](https://docs.ansible.com/projects/ansible/latest/dev_guide/testing_running_locally.html#commands) - [Ansible Lint Documentation - Homepage](https://docs.ansible.com/projects/lint/) ### Ansible Test The de facto collection testing tool since Ansible 2.9 and the initial breakout of collections from `ansible-core`, known back then as `ansible-base`. It is used across all three test types. For collection testing ansible-test has some known limitations which make the maintainer experience for testing collections with it challenging. Some examples include: 1. The requirement for the `ansible_collections` directory so that ansible-test can discover the collection for testing. You can see an example in the [Ansible documentation](https://docs.ansible.com/projects/ansible/latest/dev_guide/testing_running_locally.html#testing-an-ansible-collection) and was initially discussed in [ansible/ansible/issues/60215](https://github.com/ansible/ansible/issues/60215) back in 2019. 2. No ability to disable certain sanity tests via the `ansible-test` configuration file: [ansible/ansible/pull/74989](https://github.com/ansible/ansible/pull/74989) 3. Allow licenses other than GPLv3+: [ansible/ansible/issues/67032](https://github.com/ansible/ansible/issues/67032) 4. No ability to ignore the shebangs sanity test for `files/` and `templates/` directories in collection roles and integration test targets: [ansible/ansible/pull/79700](https://github.com/ansible/ansible/pull/79700) One feature to note with `ansible-test` is its support for running unit and integration tests via the `--docker` flag, utilizing the container images maintained by the Ansible core team: - [Quay](https://quay.io/repository/ansible/base-test-container) - base-test-container - [Quay](https://quay.io/repository/ansible/default-test-container) - default-test-container - [Quay](https://quay.io/repository/ansible/alpine-test-container) - alpine-test-container - [Quay](https://quay.io/repository/ansible/fedora-test-container) - fedora-test-container - [Quay](https://quay.io/repository/ansible/ubuntu-test-container) - ubuntu-test-container Another aspect closely related to `ansible-test` is the Azure Pipelines (AZP) provided by Red Hat. Certain collections (such as [community.general](https://github.com/ansible-collections/community.general/tree/main/.azure-pipelines)) use AZP for running integration tests on FreeBSD, MacOS and Linux VMs. As mentioned in [Clarifying the Role of ansible-test in Collection Testing](https://forum.ansible.com/t/clarifying-the-role-of-ansible-test-in-collection-testing/45763), there have been several posts from the Ansible Core team that ansible-test is no longer receiving feature enhancements for collection testing. We are in the process of updating the Ansible documentation to clarify this: [Add note on Collection Testing with ansible-test](https://github.com/ansible/ansible-documentation/pull/3654). `ansible-test` is one of particular interest to our strategy for unifying collection testing tools. It seems like a good starting point because it no longer receives feature enhancements for collection testing. This alone seems like a strong reason to consider alternatives for collection testing. It is also common ground that affects both the Ansible community and Red Hat equally. ### Unit Testing Tools For Ansible collection content, unit testing is applicable to modules and plugins. Below are the tools for this area in the current ecosystem: | Name | Description | Location in Collection | Documentation / Project Link | | --- | --- | --- | --- | | ansible-test units | Invokes unit tests via pytest. | tests/unit | [Testing collections — Ansible Community Documentation](https://docs.ansible.com/projects/ansible/latest/dev_guide/developing_collections_testing.html#adding-unit-tests) | | pytest-ansible | A pytest plugin that enables the use of ansible in tests, enables the use of pytest as a collection unit test runner, and exposes molecule scenarios through a pytest fixture. | tests/unit | [home - Ansible Pytest Documentation](https://docs.ansible.com/projects/pytest-ansible/) | #### pytest-ansible `pytest-ansible` has several pytest [fixtures](https://github.com/ansible/pytest-ansible/blob/main/src/pytest_ansible/fixtures.py) (`ansible_module`, `ansible_adhoc`, `ansible_facts`) for interacting with Ansible via `pytest`. Furthermore, there is a pytest [fixture](https://docs.ansible.com/projects/pytest-ansible/getting_started/#molecule-scenario-integration) provided by pytest-ansible for triggering molecule scenarios. ### Integration Testing Tools For Ansible collection content, integration testing is applicable to modules, plugins, roles and playbooks. Below are the tools for this area used in the current ecosystem: | Name | Description | Location in Collection | Documentation / Project Link | | --- | --- | --- | --- | | ansible-test integration | Invoke integration tests via ansible-test integration\* | tests/integration | [Testing collections — Ansible Community Documentation](https://docs.ansible.com/projects/ansible/latest/dev_guide/developing_collections_testing.html#adding-integration-tests) | | molecule | An ansible-native testing framework for collections, playbooks, and roles with configurable workflows for testing any system or service. | extensions/molecule | [Ansible Molecule](https://docs.ansible.com/projects/molecule/) [GitHub - ansible/molecule](https://github.com/ansible/molecule) [Collection Testing - Ansible Molecule](https://docs.ansible.com/projects/molecule/getting-started-collections/) | #### Molecule Molecule has gone through some significant changes since [v25.9.0](https://github.com/ansible/molecule/releases/tag/v25.9.0) with the introduction of: - An ‘[ansible-native](https://forum.ansible.com/t/help-test-molecule-v25-9-0-rc1-pre-release/44345)’ approach to testing collections: “Treats Molecule as orchestration around “plain Ansible” artifacts: inventories, create.yml/destroy.yml/converge.yml playbooks.” - The ‘ansible-native’ approach means that the [molecule-plugins](https://github.com/ansible-community/molecule-plugins) project utilized in prior molecule versions is less relevant as the drivers which used to take care of provisioning resources can now be done via collection modules in the `create.yml` and other test sequence playbooks. - Molecule is now more ‘collection aware’ with automatic scenario discovery: ```auto $ uv run molecule list INFO Collection 'community.beszel' detected. INFO Scenarios will be used from 'extensions/molecule' INFO agent_airgap ➜ list: Executing INFO agent_airgap ➜ list: Executed: Successful INFO agent_default ➜ list: Executing INFO agent_default ➜ list: Executed: Successful INFO agent_token ➜ list: Executing INFO agent_token ➜ list: Executed: Successful INFO hub_default ➜ list: Executing INFO hub_default ➜ list: Executed: Successful Instance Name │ Driver Name │ Provisioner Name │ Scenario Name │ Created │ Converged ╶───────────────┼─────────────┼──────────────────┼───────────────┼─────────┼───────────╴ instance │ default │ ansible │ agent_airgap │ false │ false instance │ default │ ansible │ agent_default │ false │ false instance │ default │ ansible │ agent_token │ false │ false instance │ default │ ansible │ hub_default │ false │ false ``` ### Sanity Testing Tools For Ansible collection content, sanity testing is applicable to modules and plugins. There is currently only one tool in this area which is `ansible-test` and the `sanity` command. At the time of writing this post there are no alternatives. Related documentation: [Testing collections — Ansible Community Documentation](https://docs.ansible.com/projects/ansible/latest/dev_guide/developing_collections_testing.html#sanity-tests). ### Static Analysis For Ansible collection content, static analysis is applicable to modules, plugins, roles and playbooks. Below are the tools related to this area used in the current ecosystem: | Name | Description | Documentation / Project Link | | --- | --- | --- | | pycodestyle | PEP8 compliance. Used by `ansible-test sanity`. | [pycodestyle’s documentation — pycodestyle 2.14.0 documentation](https://pycodestyle.pycqa.org/en/latest/) | | pylint | Static code analyzer. Used by `ansible-test sanity`. | [Pylint 4.1.0-dev0 documentation](https://pylint.readthedocs.io/en/latest/) | | yamllint | YAML linter. Used by `ansible-test sanity` and `ansible-lint`. | [yamllint documentation](https://yamllint.readthedocs.io/en/stable/) | | ansible-lint | Linter for Ansible playbooks and roles. | [home - Ansible Lint Documentation](https://docs.ansible.com/projects/lint/) | | ruff | Python linter and formatter. Available in `antsibull-nox`. | [Ruff](https://docs.astral.sh/ruff/) | | black | Python formatter. Available in `antsibull-nox`. | [Black](https://black.readthedocs.io/en/stable/) | | flake8 | Python linter. Available in `antsibull-nox`. | [Flake8](https://flake8.pycqa.org/en/latest/) | | mypy | Static type checker. Available in `antsibull-nox`. | [mypy Documentation](https://mypy.readthedocs.io/en/stable/) | ### Other Tooling The tooling listed below is used in the ecosystem for collection testing. | Name | Description | Documentation / Project Link | | --- | --- | --- | | ansible-dev-tools (ADT) | A metapackage which installs other developer tooling for running Ansible content and testing collections. | [Ansible Development Tools Documentation](https://docs.ansible.com/projects/dev-tools/#included-packages) | | ansible-dev-environment (ade) | A pip-like installer for Ansible collections. Used by `tox-ansible`. | [Ansible Development Environment Documentation](https://docs.ansible.com/projects/dev-environment/) | | andebox | A script to assist Ansible developers by encapsulating some boilerplate tasks, especially the ability of running tests from the root directory of the collection. | [GitHub - russoz-ansible/andebox: Ansible Developer’s (tool)Box](https://github.com/russoz-ansible/andebox) | | ansible-doc | Used during `ansible-test sanity` to validate plugin docs. | [Ansible documentation](https://docs.ansible.com/projects/ansible/latest/cli/ansible-doc.html) | | antsibull-docs | Provides tooling to lint Ansible collection documentation. Used by `antsibull-nox`. Using subcommand `lint-collection-docs`. | [antsibull-docs – Ansible Documentation Build Scripts](https://docs.ansible.com/projects/antsibull-docs/) | | galaxy-importer | Builds and imports Ansible collections to Galaxy and Automation Hub. Used by Partner Engineering and `antsibull-nox`. | [GitHub - ansible/galaxy-importer](https://github.com/ansible/galaxy-importer) | | antsibull-nox | A [nox](https://nox.thea.codes/en/stable/) helper library for Ansible collections. | [Antsibull Nox Helper](https://docs.ansible.com/projects/antsibull-nox/) | | tox-ansible | A [tox](https://tox.wiki/en/4.17.2/) plugin for testing Ansible collections. | [Home - Tox Ansible Documentation](https://docs.ansible.com/projects/tox-ansible/) | ### Existing Collection Testing Pathways During my research I mapped `ansible-test` and the two primary collection testing orchestration tools (`antsibull-nox` and `tox-ansible`) into two pathways: Community and Red Hat supported. Red Hat supported contains tooling maintained by the [Red Hat Ansible DevTools team](https://forum.ansible.com/g/DevTools). The below pathway headings contain diagrams with each area, their tools and how it relates to the content in a collection. I hope that this helps visualise the pathways for those of us that are visual learners. Furthermore, as part of my research I implemented both orchestration tools in the [community.beszel](https://github.com/ansible-collections/community.beszel) collection: | Pathway | Links | | --- | --- | | Community (`antsibull-nox`) | Tag: [1.0.0](https://github.com/ansible-collections/community.beszel/tree/1.0.0) [GitHub Actions Workflow](https://github.com/ansible-collections/community.beszel/actions/runs/25659452547) | | Red Hat Supported (`tox-ansible`) | Branch: [tox-ansible](https://github.com/dbrennand/community.beszel/tree/tox-ansible) PR: [#40](https://github.com/ansible-collections/community.beszel/pull/40) [GitHub Actions Workflow](https://github.com/ansible-collections/community.beszel/actions/runs/25519787309) | #### Community Pathway (Traditional) The diagram below shows the ‘traditional’ approach to collection testing, using ansible-test without any orchestration tooling. This is essentially the de facto standard in the ecosystem with many collections using the [ansible-test-gh-action](https://github.com/ansible-community/ansible-test-gh-action). At the time of writing, any new collections created from the [collection\_template](https://github.com/ansible-collections/collection_template/blob/main/.github/workflows/ansible-test.yml) uses this GitHub action. ![Screenshot From 2026-05-11 17-17-16](https://us1.discourse-cdn.com/ansible/original/2X/9/9dfb3fd3c772ebc70ceb14297e32c00b8c78ea8e.png) #### Community Pathway (`antsibull-nox`) The diagram below shows the `antsibull-nox` collection testing pathway. Maintained by Felix Fontain, `antsibull-nox` provides `nox` sessions to run different tools in the ecosystem, some of the sessions are enabled by default, others are opt-in. Generally, it provides a lot of flexibility to collection maintainers. Some key areas to highlight are that: - It relies on `ansible-test` for the sanity, unit and integration sessions. - The sanity, unit and integration sessions use the `--docker` flag with ansible-test which requires a container engine such as Docker or Podman to be installed. - `antsibull-nox` takes care of configuring the `ansible_collections` path for `ansible-test` and generating the matrices of Python versions and `ansible-core` versions supported by a collection. - Support was recently [added](https://github.com/ansible-community/antsibull-nox/pull/187) to run `molecule` as an alternative integration session. - It has support for linting collection documentation and several static analysis linters. ![Screenshot From 2026-05-11 17-19-10](https://us1.discourse-cdn.com/ansible/original/2X/d/d3db512e8b56ed225e81fe5469b17074c02c6b56.png) #### **Red Hat Supported (`tox-ansible`)** The diagram below shows the `tox-ansible` collection testing pathway. This pathway uses Red Hat supported tooling from the [Red Hat Ansible DevTools team](https://forum.ansible.com/g/DevTools). `tox-ansible` provides `tox` environments to run unit, integration, sanity tests and test validity of the collection artifact via `galaxy-importer`. Some key areas to highlight are: - The tool for integration testing is `molecule`. - The tool for unit testing is `pytest` and the `pytest-ansible` plugin. - The tool for sanity testing is `ansible-test sanity`. - During discussions with the DevTools team it was identified that this is the area which has been the most difficult to break away from due to how closely coupled some of the `ansible-test sanity` checks (such as `validate-modules`) are to `ansible-core` internals. - `ansible-dev-environment` is used to install all Python and Ansible collection dependencies required for the environments. - `tox-ansible` and `ansible-dev-environment` favor [uv](https://docs.astral.sh/uv/) for creating the Python virtual environment used for testing, downloading the required Python version for the matrices of tests and installing dependencies. During my implementation of this pathway in the [community.beszel](https://github.com/ansible-collections/community.beszel/pull/40) collection I identified several points of friction with this pathway and raised GitHub issues for these: - ✅ [Documentation: User Guide References Non Existent GitHub Action Workflow · Issue #548](https://github.com/ansible/tox-ansible/issues/548) - ✅ [Bug: Molecule package missing from Tox Integration Environment install\_deps · Issue #549](https://github.com/ansible/tox-ansible/issues/549) - ✅ [Bug: Ansible Integration and Unit Test Collection dependencies are not discovered automatically · Issue #550](https://github.com/ansible/tox-ansible/issues/550) - ✅ [Patching ansible.cfg causes exit status 2 · Issue #349](https://github.com/ansible/ansible-dev-environment/issues/349) I also created these issues downstream for tracking and I plan to submit bugfixes and documentation updates for these. ![Screenshot From 2026-05-13 16-27-48](https://us1.discourse-cdn.com/ansible/original/2X/f/fac256de18b7b0e095fc03d14b92818bbd54d745.png) ---
**Author:** ![felixfontein](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/felixfontein/32/7530_2.png) [@felixfontein](https://forum.ansible.com/u/felixfontein)\ **Post date:** [May 14, 2026, 4:23pm UTC](https://forum.ansible.com/t/unified-collection-testing-strategy-kick-off-landscape-overview/45799/2 "2026-05-14T16:23:45Z")
> [@dbrennand](#): > > Certain collections (such as [community.general](https://github.com/ansible-collections/community.general/tree/main/.azure-pipelines)) use AZP for running all three test types on FreeBSD, MacOS and Linux VMs. These VMs are only used for integration tests, not for unit or sanity tests. Unit and sanity tests are (or should be) system-agnostic. ---
**Author:** ![dbrennand](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/dbrennand/32/7830_2.png) [@dbrennand](https://forum.ansible.com/u/dbrennand)\ **Post date:** [May 14, 2026, 4:23pm UTC](https://forum.ansible.com/t/unified-collection-testing-strategy-kick-off-landscape-overview/45799/3 "2026-05-14T16:23:45Z")
Hey @felixfontein - Thanks for the clarification on that! 🙏 I’ve updated the original post to correct this. ---
**Author:** ![kks](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/kks/32/10352_2.png) [@kks](https://forum.ansible.com/u/kks)\ **Post date:** [May 14, 2026, 7:26pm UTC](https://forum.ansible.com/t/unified-collection-testing-strategy-kick-off-landscape-overview/45799/4 "2026-05-14T19:26:16Z")
This is very thorough and well written, thank you @dbrennand! Only two questions are not answered from my point of view: What is a unit test in ansible ecosystem? What is an integration test in ansible ecosystem? I am certain that it seems like answers are obvous, and they are, problem is that they are different for different people. I’ve personally run unit tests with molecule and integration tests using pytest. These are just tools, they do not determine what tests (unit, integration, etc) are executed. ---
**Author:** ![kks](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/kks/32/10352_2.png) [@kks](https://forum.ansible.com/u/kks)\ **Post date:** [May 14, 2026, 7:35pm UTC](https://forum.ansible.com/t/unified-collection-testing-strategy-kick-off-landscape-overview/45799/5 "2026-05-14T19:35:58Z")
> [@felixfontein](#): > > Unit and sanity tests are (or should be) system-agnostic. Why? I could think of a unit tests - make sure that certain crypto lib (or other non python dependency) behaves the same on all supported OS versions, ansible-core versions, python versions. This test is clearly is not system agnostic. Another example - integration tests for plugin (not module, so is only executed on controller) that depends only on python. Any integration test for this kind of plugin does not depend on system. My point is that there are unit tests that should be executed on varios platform combinations (ansible-core version, python version, os version) and integration tests that need execution only with one ansible-core version. These are completely orthogonal. ---
**Author:** ![felixfontein](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/felixfontein/32/7530_2.png) [@felixfontein](https://forum.ansible.com/u/felixfontein)\ **Post date:** [May 15, 2026, 7:36am UTC](https://forum.ansible.com/t/unified-collection-testing-strategy-kick-off-landscape-overview/45799/6 "2026-05-15T07:36:53Z")
> [@kks](#): > > Only two questions are not answered from my point of view: > What is a unit test in ansible ecosystem? > What is an integration test in ansible ecosystem? I was about to reply to these yesterday when my laptop froze and I had to hard-reboot it, and then I forgot to come back here 🙂 I think this is one of the big points - it depends also a lot on what kind of content you are testing. For plugins and modules (I’m assuming Python - for PowerShell / C# it might be slightly different) unit and integration testing means something very different than for roles or playbooks. My comment “Unit and sanity tests are (or should be) system-agnostic” is meant for plugins and (Python) modules. The idea of unit testing is to test small components of code in isolation, and generally this should _not_ depend on the operating system, but only on the ansible-core and Python verisons, and maybe on versions of dependencies. On the other hand, integration tests need to be executed on different operating systems, since they are end-to-end tests and often depend on operating system details. > [@kks](#): > > I could think of a unit tests - make sure that certain crypto lib (or other non python dependency) behaves the same on all supported OS versions, ansible-core versions, python versions. This test is clearly is not system agnostic. But in that case it’s more an integration test and not a unit test. Unit tests test components in isolation. Integration tests test components in combination. (And yes, in practice most unit tests aren’t strictly unit tests, but do also test things in combination. I still disagree that unit tests should depend on the OS, unless in extremely specialized cases; depending on versions of specific dependencies is obviously another issue, but strictly that also wouldn’t be a unit test, but an integration test.) ---
**Author:** ![dbrennand](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/dbrennand/32/7830_2.png) [@dbrennand](https://forum.ansible.com/u/dbrennand)\ **Post date:** [May 15, 2026, 12:49pm UTC](https://forum.ansible.com/t/unified-collection-testing-strategy-kick-off-landscape-overview/45799/7 "2026-05-15T12:49:23Z")
Hey @kks Thanks for reading the post and taking the time to respond 🙂 > [@kks](#): > > I’ve personally run unit tests with molecule and integration tests using pytest. Thanks for sharing that you use `molecule` and `pytest`. Is that in [GitHub - redhat-sap/rh\_operations: Ansible collection for SAP operations. · GitHub](https://github.com/redhat-sap/rh_operations)? Do you use the `pytest-ansible` plugin at all as part of your unit test suite? Also, do you use `tox-ansible` out of interest? ---
**Author:** ![kks](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/kks/32/10352_2.png) [@kks](https://forum.ansible.com/u/kks)\ **Post date:** [May 15, 2026, 1:34pm UTC](https://forum.ansible.com/t/unified-collection-testing-strategy-kick-off-landscape-overview/45799/8 "2026-05-15T13:34:48Z")
> [@dbrennand](#): > > Thanks for sharing that you use `molecule` and `pytest`. Is that in [GitHub - redhat-sap/rh\_operations: Ansible collection for SAP operations. · GitHub](https://github.com/redhat-sap/rh_operations)? > > Do you use the `pytest-ansible` plugin at all as part of your unit test suite? > Also, do you use `tox-ansible` out of interest? Unfortunately tests for that collection are not public. Yes, I mean that collection. Some examples also available here [GitHub - sap-linuxlab/community.sap\_ha\_cluster\_qa: Ansible collection HA Solutions for SAP environments QA · GitHub](https://github.com/sap-linuxlab/community.sap_ha_cluster_qa) No I do not use pytest-ansible, nor tox-ansible. What I use for linting (static checks): yaml-lint ansible-lint [ansible-later](https://github.com/thegeeklab/ansible-later) linting collection docs with antsibull-docs linting changelogs flake8 (i know there is flake in sanity, it is also run, here I can control flake more precicely) [bandit](https://bandit.readthedocs.io/en/latest/) [reuse-lint](https://reuse.readthedocs.io/en/stable/man/reuse-lint.html) shellcheck building collection documentation building collection tar file codespell galaxy-importer ansible-test sanity What I use to test modules and plugins: pytest - I run it “manually” with help of tox and makefiles for different python version and ansible-core versions. Pytest is used to run both unit tests and integration tests (there are python integration tests that do not depend on ansible-core). ansible playbooks - for plugins and some modules they are executed on localhost with controlled python and ansible-core versions (using tox, not ansible-tox). Just running them locally in tox controlled environment (ansible-core and python versions pinned). This is only for plugins and \*\_info modules. molecule - tests are executed in containers, or using cloud provisioned VMs, yes I run molecule tests for modules, not only for roles. What I use to test ansible roles and playbooks: molecule - tests are executed in containers and on cloud VMs. There are collection wide molecule tests (for instance for collection playbooks and when several roles are integration tested) as well as role specific molecule scenarios for some roles. ansible-test integration tests - this was just created in the beginning, not used anymore - I do not see any pros. ansible playbooks in roles//tests folder - these are executed on localhost in tox controlled environment. These are test playbooks that do not actually change anything when are executed. I do not have distinction between unit tests and integration tests. But yes, some tests are named “unit” and “integration” in the repo. Distinction I have is between slow tests and fast tests. I run fast tests as often as I think is appropriate. All the tests (fast and slow) run for each pr - some molecule tests (for roles and playbooks) do not run if role has not been changed by pr. Molecule tests involving cloud VMs tend to take the longest time - that is why I’ve optimized for it. If molecule tests are not involved, CI finishes in less than 3 minutes. With molecule – up to 40 minutes. ---
**Author:** ![kks](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/kks/32/10352_2.png) [@kks](https://forum.ansible.com/u/kks)\ **Post date:** [May 15, 2026, 1:38pm UTC](https://forum.ansible.com/t/unified-collection-testing-strategy-kick-off-landscape-overview/45799/9 "2026-05-15T13:38:26Z")
I forgot to mention [GitHub - foundata/ansible-docsmith: Automating Ansible role documentation (using argument\_specs.yml) · GitHub](https://github.com/foundata/ansible-docsmith) - this is a great project. There is an option to statically validate role content - [GitHub - foundata/ansible-docsmith: Automating Ansible role documentation (using argument\_specs.yml) · GitHub](https://github.com/foundata/ansible-docsmith#usage-validate) . ---
**Author:** ![mattclay](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/mattclay/32/268_2.png) [@mattclay](https://forum.ansible.com/u/mattclay)\ **Post date:** [May 15, 2026, 5:59pm UTC](https://forum.ansible.com/t/unified-collection-testing-strategy-kick-off-landscape-overview/45799/10 "2026-05-15T17:59:33Z")
It’s worth pointing out that there are some key differences to how ansible-test handles unit and integration tests compared to other tools: - ansible-test can collect and report on code coverage for integration tests. - ansible-test can run unit tests against target-only Python versions. Regarding CI support: - ansible-test supports GHA, not just AZP, for creating ephemeral test VMs. - ansible-test can provide those same VMs to authorized users outside CI. - ansible-test can selectively run tests in CI based on changes in a PR. There are also some noteworthy debugging features: - ansible-test has integrated debugger support for VS Code and PyCharm for both Python and PowerShell code. - Debugging can be used via manual invocation of ansible commands, as well as when running integration tests, even when running on a remote host or in a container. - Use on both plugins and modules is supported, including setting breakpoints, stepping, setting/inspecting variables, etc. Also, a few clarifications and corrections: - The statement “no longer receiving feature enhancements for collection testing” may be a bit misleading – new features continue to be added to ansible-test that benefit collections, just usually not ones that are exclusively for collections with no benefit to ansible-core (although we still do make such changes occasionally). - The statement “Invoke integration tests via ansible-test sanity.” is incorrect. The various “integration\*” commands are used to invoke integration tests. - Static analysis can be considered a subset of sanity testing – while many sanity tests perform static analysis (e.g. pylint), others execute code (e.g. validate-modules). - The initial post mentions both four test types, and three. If static analysis is rolled into sanity tests (see preceding bullet point), then this would be resolved. - When talking about “unit” tests in terms of ansible-test, these are tests written in Python and run internally using pytest. - When talking about “integration” tests in terms of ansible-test, these are tests usually written using Ansible roles, although sometimes shell scripts are also used. ---
**Author:** ![mikemorency](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/mikemorency/32/2672_2.png) [@mikemorency](https://forum.ansible.com/u/mikemorency)\ **Post date:** [May 15, 2026, 7:43pm UTC](https://forum.ansible.com/t/unified-collection-testing-strategy-kick-off-landscape-overview/45799/11 "2026-05-15T19:43:17Z")
Is there any documentation for those features? I’ve been using ansible-test daily for two years now and most of that sounds very useful. On a related note, whatever tooling is chosen for each type of test needs much better documentation. Sanity is OK right now but unit and integration tests are a bit of a mystery IMO. So frustrating sinking hours into getting tests to work because of an issue completely unrelated to your collection. Since there’s so many different “things” people are testing, it seems like a simple but flexible tool that is easy to understand would be valuable ---
**Author:** ![mattclay](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/mattclay/32/268_2.png) [@mattclay](https://forum.ansible.com/u/mattclay)\ **Post date:** [May 15, 2026, 10:57pm UTC](https://forum.ansible.com/t/unified-collection-testing-strategy-kick-off-landscape-overview/45799/12 "2026-05-15T22:57:36Z")
Which features are you looking for documentation on? ---
**Author:** ![mikemorency](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/mikemorency/32/2672_2.png) [@mikemorency](https://forum.ansible.com/u/mikemorency)\ **Post date:** [May 15, 2026, 11:21pm UTC](https://forum.ansible.com/t/unified-collection-testing-strategy-kick-off-landscape-overview/45799/13 "2026-05-15T23:21:11Z")
> [@mattclay](#): > > - ansible-test can collect and report on code coverage for integration tests. > > - ansible-test supports GHA, not just AZP, for creating ephemeral test VMs. > > - ansible-test can provide those same VMs to authorized users outside CI. > > - ansible-test can selectively run tests in CI based on changes in a PR. > > - ansible-test has integrated debugger support for VS Code and PyCharm for both Python and PowerShell code. These features ---
**Author:** ![mattclay](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/mattclay/32/268_2.png) [@mattclay](https://forum.ansible.com/u/mattclay)\ **Post date:** [May 16, 2026, 12:23am UTC](https://forum.ansible.com/t/unified-collection-testing-strategy-kick-off-landscape-overview/45799/14 "2026-05-16T00:23:20Z")
> ansible-test can selectively run tests in CI based on changes in a PR This feature is not covered in the official documentation. It is enabled by using the `--changed` option. That option works locally (with caveats), and on AZP. It is not yet supported on GHA, although it should be possible to implement it. To see it in use, take a look at the `community.general` collection. Here’s where it is enabled: [community.general/tests/utils/shippable/shippable.sh at 47a19fad1bd23cd1d358d1538346bf9a977ed570 · ansible-collections/community.general · GitHub](https://github.com/ansible-collections/community.general/blob/47a19fad1bd23cd1d358d1538346bf9a977ed570/tests/utils/shippable/shippable.sh#L125) > ansible-test can collect and report on code coverage for integration tests. There is brief mention of this feature in the official documentation: [Testing Ansible and Collections — Ansible Community Documentation](https://docs.ansible.com/projects/ansible/latest/dev_guide/testing_running_locally.html#code-coverage) To see it in use, take a look at the `community.general` collection. Here’s where it is enabled: [community.general/tests/utils/shippable/shippable.sh at 47a19fad1bd23cd1d358d1538346bf9a977ed570 · ansible-collections/community.general · GitHub](https://github.com/ansible-collections/community.general/blob/47a19fad1bd23cd1d358d1538346bf9a977ed570/tests/utils/shippable/shippable.sh#L101-L115) NOTE: This feature can also be used locally. It doesn’t require CI or using a service like Codecov. It also works for unit and sanity tests (although only the `import` sanity test contributes to coverage). > ansible-test supports GHA, not just AZP, for creating ephemeral test VMs. This feature is not covered in the official documentation, and is a relatively new feature. It is currently limited to specific projects within the `ansible-collections` and `redhat-cop` GitHub organizations, but we’re open to considering requests for usage in additional projects. To see it in use, take a look at the `amazon.aws` collection. Here’s where it is enabled: [amazon.aws/.github/workflows/integration.yml at c0bc9f945fbb94259b796077e23c37662358095c · ansible-collections/amazon.aws · GitHub](https://github.com/ansible-collections/amazon.aws/blob/c0bc9f945fbb94259b796077e23c37662358095c/.github/workflows/integration.yml#L143) NOTE: This collection uses it for ephemeral AWS credentials (another feature I forgot to mention), rather than requesting a VM directly. > ansible-test can provide those same VMs to authorized users outside CI. This is briefly mentioned in the official documentation: [Testing Ansible and Collections — Ansible Community Documentation](https://docs.ansible.com/projects/ansible/latest/dev_guide/testing_running_locally.html#remote-virtual-machines) As the documentation notes, users outside of authorized AZP (and now GHA) projects will require an API key to take advantage of this feature from their local system. Maintainers of collections authorized to use the feature in CI can request access for use on their local system. > ansible-test has integrated debugger support for VS Code and PyCharm for both Python and PowerShell code. This feature is not covered in the official documentation, and is still considered experimental. As such, it doesn’t show up in the `--help` output for users unless they have the previously mentioned API key. However, despite that, any user can still use the feature. The best reference for the options at the moment is to look at the code: [ansible/test/lib/ansible\_test/\_internal/cli/environments.py at 405a6052b0238d5c1bbd956e83e4d12a0f692d51 · ansible/ansible · GitHub](https://github.com/ansible/ansible/blob/405a6052b0238d5c1bbd956e83e4d12a0f692d51/test/lib/ansible_test/_internal/cli/environments.py#L451-L486) ---
**Author:** ![felixfontein](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/felixfontein/32/7530_2.png) [@felixfontein](https://forum.ansible.com/u/felixfontein)\ **Post date:** [May 16, 2026, 7:29am UTC](https://forum.ansible.com/t/unified-collection-testing-strategy-kick-off-landscape-overview/45799/15 "2026-05-16T07:29:49Z")
> [@mattclay](#): > > > ansible-test can selectively run tests in CI based on changes in a PR > > This feature is not covered in the official documentation. It is enabled by using the `--changed` option. That option works locally (with caveats), and on AZP. It is not yet supported on GHA, although it should be possible to implement it. > > To see it in use, take a look at the `community.general` collection. Here’s where it is enabled: [community.general/tests/utils/shippable/shippable.sh at 47a19fad1bd23cd1d358d1538346bf9a977ed570 · ansible-collections/community.general · GitHub](https://github.com/ansible-collections/community.general/blob/47a19fad1bd23cd1d358d1538346bf9a977ed570/tests/utils/shippable/shippable.sh#L125) This feature is also exposed by [GitHub - ansible-community/ansible-test-gh-action: A composite GitHub Action encapsulating the GitHub Actions CI/CD workflows setup necessary for testing Ansible collection repositories on GitHub · GitHub](https://github.com/ansible-community/ansible-test-gh-action) (`pull-request-change-detection` option) and by antsibull-nox ([Change detection - antsibull-nox – Antsibull Nox Helper](https://docs.ansible.com/projects/antsibull-nox/change-detection/)). > [@mattclay](#): > > > ansible-test supports GHA, not just AZP, for creating ephemeral test VMs. > > This feature is not covered in the official documentation, and is a relatively new feature. It is currently limited to specific projects within the `ansible-collections` and `redhat-cop` GitHub organizations, but we’re open to considering requests for usage in additional projects. > > To see it in use, take a look at the `amazon.aws` collection. Here’s where it is enabled: [amazon.aws/.github/workflows/integration.yml at c0bc9f945fbb94259b796077e23c37662358095c · ansible-collections/amazon.aws · GitHub](https://github.com/ansible-collections/amazon.aws/blob/c0bc9f945fbb94259b796077e23c37662358095c/.github/workflows/integration.yml#L143) > > NOTE: This collection uses it for ephemeral AWS credentials (another feature I forgot to mention), rather than requesting a VM directly. Interestingly amazon.aws does not use VMs (the ansible-community/ansible-test-gh-action action it uses doesn’t support the `--remote` flag). It “only” uses authorization for setting up a new AWS instance to use for the tests (I guess through ansible-test’s AWS cloud plugin). ---
**Author:** ![felixfontein](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/felixfontein/32/7530_2.png) [@felixfontein](https://forum.ansible.com/u/felixfontein)\ **Post date:** [May 16, 2026, 7:35am UTC](https://forum.ansible.com/t/unified-collection-testing-strategy-kick-off-landscape-overview/45799/16 "2026-05-16T07:35:37Z")
Change detection and code coverage are likely _the_ `ansible-test unit` and `ansible-test integration` features that are not covered by any other unit/integration test runner I’m aware of (except by wrapper runners like antsibull-nox, which fall back to ansible-test for the heavy lifting). Split-controller testing and providing proper isolation of both controller and target is another big feature of `ansible-test integration` that I’m not sure is supported by other integration test runners either. (This also means that user-provided code always runs in ephemeral VMs or containers, and not directly in the CI environment. So this isn’t just useful to prevent unexpected changes, but also useful from a security point of view.) ---
**Author:** ![dbrennand](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/dbrennand/32/7830_2.png) [@dbrennand](https://forum.ansible.com/u/dbrennand)\ **Post date:** [May 16, 2026, 3:16pm UTC](https://forum.ansible.com/t/unified-collection-testing-strategy-kick-off-landscape-overview/45799/17 "2026-05-16T15:16:56Z")
Hey @mattclay - Thanks for the detailed response on `ansible-test` and the clarifications. > [@mattclay](#): > > ansible-test supports GHA, not just AZP, for creating ephemeral test VMs. Are you referring to [GitHub - ansible-community/ansible-test-gh-action: A composite GitHub Action encapsulating the GitHub Actions CI/CD workflows setup necessary for testing Ansible collection repositories on GitHub · GitHub](https://github.com/ansible-community/ansible-test-gh-action) ? - It is included in the ‘Traditional’ Community pathway diagram, or you mean some other functionality? EDIT: Nevermind, seen you covered this in: [Unified Collection Testing Strategy - Kick off & Landscape Overview - #14 by mattclay](https://forum.ansible.com/t/unified-collection-testing-strategy-kick-off-landscape-overview/45799/14) > [@mattclay](#): > > The statement “Invoke integration tests via ansible-test sanity.” is incorrect. The various “integration\*” commands are used to invoke integration tests. Yes, that is a typo error, it should say `ansible-test integration` - I’ll correct the post thanks for spotting that! > [@mattclay](#): > > When talking about “integration” tests in terms of ansible-test, these are tests usually written using Ansible roles, although sometimes shell scripts are also used. Yes! That’s a good point to mention actually. `molecule` does have similarities to this. It can also use roles in the test sequences, but you can also use playbooks and have ansible inventory too for test scenarios. > [@mattclay](#): > > - Static analysis can be considered a subset of sanity testing – while many sanity tests perform static analysis (e.g. pylint), others execute code (e.g. validate-modules). > - The initial post mentions both four test types, and three. If static analysis is rolled into sanity tests (see preceding bullet point), then this would be resolved. Ah, that will be because when I was initially drafting this I did have the static analysis tools under sanity testing (“the subset”). However, I then separated it out for the same reason you’ve highlighted. Those are tools performing static analysis and not executing code (e.g., ansible-test sanity’s `validate-modules`). ---
**Author:** ![dbrennand](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/dbrennand/32/7830_2.png) [@dbrennand](https://forum.ansible.com/u/dbrennand)\ **Post date:** [May 16, 2026, 4:35pm UTC](https://forum.ansible.com/t/unified-collection-testing-strategy-kick-off-landscape-overview/45799/18 "2026-05-16T16:35:41Z")
> [@felixfontein](#): > > Change detection and code coverage are likely _the_ `ansible-test unit` and `ansible-test integration` features that are not covered by any other unit/integration test runner I’m aware of (except by wrapper runners like antsibull-nox, which fall back to ansible-test for the heavy lifting). Hey @felixfontein - Whilst reviewing the `pytest-ansible` source I did notice an option for running `molecule` scenarios called `skip_no_git_change`: [pytest-ansible/src/pytest\_ansible/molecule.py at main · ansible/pytest-ansible · GitHub](https://github.com/ansible/pytest-ansible/blob/main/src/pytest_ansible/molecule.py#L201-L218) I can’t find any documentation on this option (_sigh…_) but it looks to skip scenarios when no changes are detected via `git diff` to the role. However, it appears to be outdated given the newest convention for molecule scenario location inside a collection is `extensions/molecule`. It’s looking for molecule scenarios inside the role directory itself (e.g., `roles//molecule//molecule.yml`). Regarding code coverage, I wanted to see if I could get `pytest-cov` working with `tox-ansible` and I have managed to with the use of `.coveragerc` and slight modification to the `tox-ansible.ini` file: [Implement tox-ansible for collection testing by dbrennand · Pull Request #40 · ansible-collections/community.beszel · GitHub](https://github.com/ansible-collections/community.beszel/pull/40/changes/a7013204a68e9fff2b1a2b01d002bb7ad33c0982) It would be nice if this could be a supported feature out of the box, say like a `--coverage` argument: ```auto unit-py3.10-2.17: commands[0]> python3 -m pytest --cov --cov-config=.coveragerc --ansible-unit-inject-only ./tests/unit ======================================================================== test session starts ========================================================================= platform darwin -- Python 3.10.20, pytest-9.0.3, pluggy-1.6.0 cachedir: .tox/unit-py3.10-2.17/.pytest_cache ansible: 2.17.14 rootdir: /Users/dab/github.com/dbrennand/community.beszel configfile: pyproject.toml plugins: ansible-26.4.0, cov-7.1.0, xdist-3.8.0, anyio-4.13.0 collected 26 items tests/unit/plugins/modules/test_system.py .......... [38%] tests/unit/plugins/modules/test_system_info.py .... [53%] tests/unit/plugins/modules/test_universal_token.py ............ [100%] =========================================================================== tests coverage =========================================================================== _________________________________________________________coverage: platform darwin, python 3.10.20-final-0__________________________________________________________ Name Stmts Miss Cover Missing ------------------------------------------------------------------------ plugins/module_utils/pocketbase_utils.py 33 25 24% 9-12, 17-23, 27-36, 40-49 plugins/modules/system.py 101 22 78% 176-178, 208-209, 232, 263-264, 273-274, 289-299, 313-314, 369-370, 393-394, 406 plugins/modules/system_info.py 38 7 82% 127-129, 153, 174-175, 191 plugins/modules/universal_token.py 52 5 90% 109-111, 145, 202 ------------------------------------------------------------------------ TOTAL 224 59 74% ========================================================================= 26 passed in 0.30s ========================================================================= unit-py3.10-2.17: OK (10.56=setup[1.46]+cmd[7.61,0.41,1.09] seconds) congratulations :) (11.05 seconds) ``` ---
**Author:** ![jborean](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/jborean/32/1525_2.png) [@jborean](https://forum.ansible.com/u/jborean)\ **Post date:** [May 18, 2026, 1:39am UTC](https://forum.ansible.com/t/unified-collection-testing-strategy-kick-off-landscape-overview/45799/19 "2026-05-18T01:39:47Z")
One other component missed here is support for Windows target and PowerShell. For sanity tests we support the `pslint` target which does some static analysis on PowerShell module and module\_utils. It runs on the “controller” and not the Windows target using `pwsh`. Each Ansible release ties itself to a specific PowerShell 7.x version as well as a specific `PSScriptAnalyzer` version to ensure that future updates don’t invalidate existing rules or pick up errors added in new rules or bugfixes. There is also `pwsh` support for the `validate-modules` test target that is used to validate the arg spec between a PowerShell module and what is in the documentation. Like the `pslint` sanity test this also uses a local `pwsh` install to run the code needed to export the module arg spec. The `ansible-test windows-integration` command can be used to run integration test targets that start with `win_` or have the `windows` entry in the target’s `aliases` file. Unlike `integration` you can provider your own Windows target through the `tests/integration/inventory.winrm` file or if you have access to the core-ci keys @mattclay has mentioned can request a short lived Windows host that `ansible-test` will provision. While the inventory file is called `inventory.winrm` that is mostly a historical naming scheme and you are not restricted to only the `winrm` connection plugin but any plugin that can connect to a Windows target. If we are aiming to have something replace the functionality done with `ansible-test` then at a minimum it should also support the `pslint` target as well as the PowerShell module integration done by `validate-modules` (arg spec validation). For integration testing being able to provide your own inventory as a test target on Windows targets would also be required but host provisioning could potentially still be done through another tool to replicate the `core-ci` behaviour in `ansible-test`. ---
**Author:** ![dbrennand](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/dbrennand/32/7830_2.png) [@dbrennand](https://forum.ansible.com/u/dbrennand)\ **Post date:** [May 18, 2026, 9:13am UTC](https://forum.ansible.com/t/unified-collection-testing-strategy-kick-off-landscape-overview/45799/20 "2026-05-18T09:13:34Z")
> [@jborean](#): > > If we are aiming to have something replace the functionality done with `ansible-test` then at a minimum it should also support the `pslint` target as well as the PowerShell module integration done by `validate-modules` (arg spec validation). Hey @jborean - Thanks for your response on this. RE: The `pslint` target in `ansible-test sanity` - It runs PSScriptAnalyzer only, right? Just collecting a few links here related to it: - [ansible/test/lib/ansible\_test/\_internal/commands/sanity/pslint.py at devel · ansible/ansible · GitHub](https://github.com/ansible/ansible/blob/devel/test/lib/ansible_test/_internal/commands/sanity/pslint.py#L49) - [ansible/test/lib/ansible\_test/\_data/requirements/sanity.pslint.ps1 at devel · ansible/ansible · GitHub](https://github.com/ansible/ansible/blob/devel/test/lib/ansible_test/_data/requirements/sanity.pslint.ps1) - [ansible/test/lib/ansible\_test/\_util/controller/sanity/pslint/pslint.ps1 at devel · ansible/ansible · GitHub](https://github.com/ansible/ansible/blob/devel/test/lib/ansible_test/_util/controller/sanity/pslint/pslint.ps1) - [ansible/test/lib/ansible\_test/\_util/controller/sanity/pslint/settings.psd1 at devel · ansible/ansible · GitHub](https://github.com/ansible/ansible/blob/devel/test/lib/ansible_test/_util/controller/sanity/pslint/settings.psd1) - [GitHub - jborean93/PSSA-PSCustomUseLiteralPath: PSScriptAnalyzer rule for detecting the usage of -Path over -LiteralPath · GitHub](https://github.com/jborean93/PSSA-PSCustomUseLiteralPath) I found another project on GitHub called `pslint` but I don’t think this sanity target in `ansible-test` has any relation to that. > [@jborean](#): > > For integration testing being able to provide your own inventory as a test target on Windows targets would also be required but host provisioning could potentially still be done through another tool to replicate the `core-ci` behaviour in `ansible-test`. With the latest changes with `molecule` and being `ansible-native’ - You can roll your own inventory and in theory be able to provision a Windows VM on a public cloud. However, I don’t know of any collection doing this. I’ve asked DevTools if they know of any example. Wouldn’t the above scenario be applicable more to testing modules on PowerShell 5 (Desktop) right? As that requires Windows, but for PowerShell 6+ could a container with `pwsh` binary installed such as ones provided by Microsoft be sufficient? Have you ever tested that? [Next page](https://forum.ansible.com/t/unified-collection-testing-strategy-kick-off-landscape-overview/45799.md?page=2)