# Integrating \`ansible-test integration\` (no pun intended) and \`molecule\` **URL:** \ **Category:** Project Discussions\ **Tags:** ansible-test, molecule, community-openwrt\ **Created:** [December 22, 2025, 12:44am UTC](https://forum.ansible.com/t/integrating-ansible-test-integration-no-pun-intended-and-molecule/45018 "2025-12-22T00:44:06Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![russoz](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/russoz/32/127_2.png) [@russoz](https://forum.ansible.com/u/russoz)\ **Post date:** [December 22, 2025, 12:44am UTC](https://forum.ansible.com/t/integrating-ansible-test-integration-no-pun-intended-and-molecule/45018/1 "2025-12-22T00:44:06Z")
Hi everyone, As [advertised before](https://forum.ansible.com/t/announcement-community-openwrt/44964/2), we have been working on community.openwrt for the past couple of weeks. The role that this collection is derived of, `gekmihesg.openwrt` had some `molecule` tests that relied on docker images for OpenWrt that are published regularly by the project. Note that by it’s very nature (a specific platform), the most reliable way to test the collection is against those container images. That setup was carried over to the collection (with some adjustments, of course), it lives now in the `extensions/molecule` directory as it’s for for `molecule` running on collections, and it is working smoothly. However, we would like to have integration tests for each module, so we have started with one module, `command`, as is quite a simple one. To get that working, we are using `runme.sh` to trigger the `molecule` run. There is a PR open for that, and we would appreciate having more feedback. > > > \##### SUMMARY > > First attempt at running integration tests using \`molecule\`. > > … > > > \##### ISSUE TYPE > > \- Test Pull Request > > \##### COMPONENT NAME > > command I am not entirely happy with the mechanics of it, but so far I could not find a better way to make it work. PS: I have already asked for feedback in Matrix, but it occurred to me that there is probably(?!) more people in the forum than in the chat. Sorry for the cross-posting. ---
**Author:** ![kks](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/kks/32/10352_2.png) [@kks](https://forum.ansible.com/u/kks)\ **Post date:** [December 22, 2025, 11:02am UTC](https://forum.ansible.com/t/integrating-ansible-test-integration-no-pun-intended-and-molecule/45018/2 "2025-12-22T11:02:06Z")
What is the reason you want molecule tests to be “integrated” into integration tests? I just run molecule tests separately. I have unit tests (python + ansible), integration tests and molecule tests - these are different test categories, I do not see benefit of mixing them. Do you? One benefit of separating molecule and integration tests might be not to run all the molecule tests all the time - they tend to be costly (take a lot of time). You can run molecule tests for some modules or roles only if respective modules or roles are changed in the PR. just name the molecule scenario as the role/module for simplicity. some comments to the code " ansible.builtin.include\_tasks: ../../tasks/main.yml" “cp -a “$TARGET\_DIR/tasks/.” “$ROOT/tasks/”” you do not have to copy tasks - just create ‘roles’ folder inside your scenario and put you tasks (they look like a role) there and include them as role - ansible will look for roles to import / inlcude in ‘roles’ folder by default. I just create separate collection for molecule playbooks and roles and import all the content from there. This simplifies the reuse. But I use completely different repo structure (non standard) that allows that. “cp -a “$TARGET\_DIR/molecule” “$MOL\_DIR”” why do you copy molecule scenarios? molecule saves all the data in ephemeral folders anyway. I know that official recommendation is to put molecule folder inside extension folder. There is actually no need to do that, you can put molecule folder inside tests folder there it belongs - as you have tests there. So all the tests are inside tests folder. ---
**Author:** ![russoz](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/russoz/32/127_2.png) [@russoz](https://forum.ansible.com/u/russoz)\ **Post date:** [December 22, 2025, 11:22pm UTC](https://forum.ansible.com/t/integrating-ansible-test-integration-no-pun-intended-and-molecule/45018/3 "2025-12-22T23:22:44Z")
Hi @kks , Thanks for your response! Well, there’s a lot to unpack there. > [@kks](#): > > What is the reason you want molecule tests to be “integrated” into integration tests? I have just written a response to that question (or close enough) in the PR itself: > > > \> What's the reasoning to use \`ansible-test\` just to call \`molecule\`? The setup …works, but it seems a bit too layered to me? Or is the idea that the tests can be run by both frameworks, \`ansible-test\` \_and\_ \`modulecule\` independently? > > In other collections (and I am totally biased towards community.general, where I have previously spent 99.8463543% of my Ansible time :smile: ), it is not only \_usual\_ but it is actually \_promoted\_ that developers create integration tests (ansible-test style) for the modules. IMO that is a very good thing, it makes sense, and it gives us the confidence that the module works on a real playbook/task list. That is the primary objective of this PR, to establish a (integration) test framework for community.openwrt modules. The fact most (if not all) collections follow the standard practices in Ansible, meaning using \`ansible-test integration\`, and having this structure already in place (e.g. \`ansible-test integration\` checks for dependencies in \`tests/integration/requirements.yml\`), are arguments in favour of using \`ansible-test\`. That's more elaborated way to say the same thing @Jonny007-MKD wrote above. This is the justification for \`ansible-test\`. > > However, in other collections, \`ansible-test\` is used passing \`--docker ubuntu2404\` (or whatever version), or \`--docker fedora42\`, or \`... alpineXX\`, etc. Also in these other collections, for some modules the test cannot simply be executed from within a container, they must run from a full-fledged VM (\`c.g.snap\` comes to mind because I worked in that module), and again, the CI workflows provide support for a range of systems - VMs being provisioned, used for testing, and destroyed. None of that makes sense in this collection because: > \* \`ansible-test\` has a list of pre-defined docker targets that makes no sense for OpenWrt > \* it allows passing a custom docker image, but it gives us very limited control of how to do that - for example, the openwrt images \_need\_ us to run some endless command, like \`sleep infinity\`, I have not tried this much but I could not make it work. > \* similarly, the infrastructure available for VMs is also not relevant for this collection, for obvious reasons. > > That's where \`molecule\` steps in: it provides an easy way to have fine-grained control of multiple containers, and we can specify the standard OpenWrt images, custom commands to execute (not just the \`sleep infinity\`), etc. > > So this mechanism, as it is, tries to get the best from both worlds. The price it pays for that is: indeed it is too layered. I don't like \_that\_ very much, but I like the fact that we'll be able to have the CI run tests for just (the modified) module, rather than run all the tests for everything, every single time. > > Sorry/not sorry for the long text. It is a very good question and I felt compelled to be thorough (even though I may have forgotten something here and there). > [@kks](#): > > " ansible.builtin.include\_tasks: ../../tasks/main.yml" > “cp -a “$TARGET\_DIR/tasks/.” “$ROOT/tasks/”” > you do not have to copy tasks - just create ‘roles’ folder inside your scenario and put you tasks (they look like a role) there and include them as role - ansible will look for roles to import / inlcude in ‘roles’ folder by default. I just create separate collection for molecule playbooks and roles and import all the content from there. This simplifies the reuse. But I use completely different repo structure (non standard) that allows that. Ha! I did not know that! It looks very promising, thanks for pointing it out. Any chance you can paste the links to the repos where you do that? Assuming they are not private, of course. That being said, I don’t entirely discard using a non-standard structure here, but I would like to avoid doing that if possible. > [@kks](#): > > “cp -a “$TARGET\_DIR/molecule” “$MOL\_DIR”” why do you copy molecule scenarios? molecule saves all the data in ephemeral folders anyway. That’s exactly to mesh them with the test tasks. But given your prior suggestion we may not need to do that at all. > [@kks](#): > > I know that official recommendation is to put molecule folder inside extension folder. There is actually no need to do that, you can put molecule folder inside tests folder there it belongs - as you have tests there. So all the tests are inside tests folder. And we do have molecule tests in the extension folder, they were the tests we added back in the day to the `gekmihesg.openwrt` role. They serve like a “quick” check the general thing is working. The structure `tests/integration/target/*`, on the other hand, is meant to hold tests matching the plugins (modules and otherwise) in the collection, one in each target. I would like very much to keep that structure - or if we are not keeping it, then we have some conceptually equivalent/similar. Last, but not least, whilst I appreciate your commenting, I would like to suggest we take discussions of code choices/details to the PR at GH. It is much easier to point the exact code we are talking, and to suggest modifications. ---
**Author:** ![russoz](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/russoz/32/127_2.png) [@russoz](https://forum.ansible.com/u/russoz)\ **Post date:** [December 23, 2025, 1:50am UTC](https://forum.ansible.com/t/integrating-ansible-test-integration-no-pun-intended-and-molecule/45018/4 "2025-12-23T01:50:15Z")
@kks I have already implemented some of your ideas in the PR. Thank you! ---
**Author:** ![kks](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/kks/32/10352_2.png) [@kks](https://forum.ansible.com/u/kks)\ **Post date:** [December 23, 2025, 8:13am UTC](https://forum.ansible.com/t/integrating-ansible-test-integration-no-pun-intended-and-molecule/45018/5 "2025-12-23T08:13:39Z")
> [@russoz](#): > > Any chance you can paste the links to the repos where you do that? > **[community.sap\_ha\_cluster\_qa/ansible\_collections/sap/cluster\_qa/roles/test02/mole...](https://github.com/sap-linuxlab/community.sap_ha_cluster_qa/tree/95e19a141b42d7e55e0822fb2a7afe74e243b2f2/ansible_collections/sap/cluster_qa/roles/test02/molecule)** > > Ansible collection HA Solutions for SAP environments QA - sap-linuxlab/community.sap\_ha\_cluster\_qa Here some examples. I assume they require explanation, I do not do things as it is done in other collections and there are reasons for that. If there are any questions or comments to below - will be glad to address them. In the link above you see tests for one role, they are placed inside the role folder to be close the the role they tests. They can be also placed in tests folder. I do not know why but i tend to put role molecule tests close to the roles, but tests for plugins and module into tests folder. As you can see here > Molecule reuses playbooks from separate collection (in the same repo) that sole purpose is to support molecule tests. This is possible due to repo structure - I do not put collection content inside repo root folder, instead I put all in ansible\_collections//\ folder This allows to set only one environment variable ANSIBLE\_COLLECTIONS\_PATH to effectively “install” (make available for ansible-galaxy to find) all the collections that are inside this repo. See [community.sap\_ha\_cluster\_qa/tox.ini at 95e19a141b42d7e55e0822fb2a7afe74e243b2f2 · sap-linuxlab/community.sap\_ha\_cluster\_qa · GitHub](https://github.com/sap-linuxlab/community.sap_ha_cluster_qa/blob/95e19a141b42d7e55e0822fb2a7afe74e243b2f2/tox.ini#L34) All other collections (that are required for testing for instance, or just dependencies for developed collection) are installed in python virtual environment that is managed by tox. Here is installation code [community.sap\_ha\_cluster\_qa/Makefile at 95e19a141b42d7e55e0822fb2a7afe74e243b2f2 · sap-linuxlab/community.sap\_ha\_cluster\_qa · GitHub](https://github.com/sap-linuxlab/community.sap_ha_cluster_qa/blob/95e19a141b42d7e55e0822fb2a7afe74e243b2f2/Makefile#L10) Yes - all the dependencies for molecule (and everything else) are managed outside of molecule (or ansible-lint). That allows to test anything with any dependency I want. For instance if molecule versions would be updated in [community.sap\_ha\_cluster\_qa/tox/requirements-molecule.txt at 95e19a141b42d7e55e0822fb2a7afe74e243b2f2 · sap-linuxlab/community.sap\_ha\_cluster\_qa · GitHub](https://github.com/sap-linuxlab/community.sap_ha_cluster_qa/blob/95e19a141b42d7e55e0822fb2a7afe74e243b2f2/tox/requirements-molecule.txt#L2) tox will ensure (for each run) that required molecule version is installed. This is complete opposite to your approach in the PR “pip install molecule ‘molecule-plugins[docker]’” - dependencies and executions are in one script that prevents to execute with different dependencies (or makes it harder). I also tend to pin all the dependencies as much as possible. Having all the depedencies in separate requirements-\*.txt files allows to use dependabot (see dependabot PRs in the cluster repo I mentioned). I might be wrong, but I think in the current setup it is not possible (or I did not get how) to run molecule tests locally from one’s laptop. I suggest you enable it. I made it possible with makefiles, see > This code allows anyone to run `make molecule - - ` - make will ensure that all dependencies are installed (python and ansible collections). All the molecule commands will be executed inside $WORKDIR folder (by default tests). Workdir folder can be changed by simply setting WORKDIR environment variable: `WORKDIR=another/folder make - - molecule --version`. This is just an example, use tools that suit your need and are familiar to you. I see you set ANSIBLE\_ROLES\_PATH and placed the role directly inside molecule scenario. ```bash export ANSIBLE_ROLES_PATH ANSIBLE_ROLES_PATH="$(pwd)" ``` If I am not mistaken ansible by default looks for roles inside `roles` folder in the current folder (molecule runs inside scenario folder). So you do not have to set this env variable, just place the roles you need for scenario inside `roles` folder. That will of course limit your ability to reuse - if you later decide to use same helper role for another scenario. In that case please set ANSIBLE\_ROLES\_PATH=“$(pwd):$ANSIBLE\_ROLES\_PATH” so you do not remove values that might be set from outside. Or just just use ANSIBLE\_COLLECTIONS\_PATH approach I described here. > [@russoz](#): > > I have just written a response to that question (or close enough) in the PR itself: > [Integration test: `command` by russoz · Pull Request #49 · ansible-collections/community.openwrt · GitHub](https://github.com/ansible-collections/community.openwrt/pull/49#issuecomment-3684435835) > [@](#): > > it is not only _usual_ but it is actually _promoted_ that developers create integration tests (ansible-test style) for the modules. Unfotunately I never got to use `ansible-test integration` command - ansible-test does too much behind the scenes. What I do not get is - CI is executed in container (github runner), that runs ansible-test - all the tests are executed inside containers. And you want tests to be molecule scenarios that are running containers. So it is container inside container inside container. I bet it is much easier (and you have more control over dependencies and their versions and less to download) to execute molecule directly - it is just a python package you can execute it anywhere. At the moment I see there are two groups of CI jobs - one for integration tests and one for molecule tests. Does it mean that molecule tests will be executed twice as molecule and as integration tests. > [@russoz](#): > > I don’t entirely discard using a non-standard structure here, but I would like to avoid doing that if possible. Try it - the only “issue” is that no one will be able to install collection directly from github - this is actually not an issue, no one should install code from the internet. Everything else is just a bonus with this structure. > [@russoz](#): > > would like to suggest we take discussions of code choices/details to the PR at GH. This is not about code at the moment, but about approach in general. You can still make changes and decisions that will make this collection not a copy of all the content that was created years ago. No disrespect to old content, but ansible changed since when. BTW if you like approach with separate collection for molecule playbooks, I’ve proposed that years ago [Switch to a single unified playbook for testing an entire scenario · Issue #3918 · ansible/molecule · GitHub](https://github.com/ansible/molecule/issues/3918#issuecomment-1611225035) (stars with “Here is suggestion I have.”). Would be nice if someone from steering commetee push for it as well. ---
**Author:** ![russoz](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/russoz/32/127_2.png) [@russoz](https://forum.ansible.com/u/russoz)\ **Post date:** [December 27, 2025, 12:00pm UTC](https://forum.ansible.com/t/integrating-ansible-test-integration-no-pun-intended-and-molecule/45018/6 "2025-12-27T12:00:19Z")
Hi @kks I gave it a look at your repo and whilst I do appreciate some of the things you did there - in fact, as mentioned above, I have incorporated some of them in that PR in `community.openwrt` - I have to disagree with your claim: > [@kks](#): > > Try it - the only “issue” is that no one will be able to install collection directly from github - this is actually not an issue, no one should install code from the internet. Everything else is just a bonus with this structure. Other issues derive from having the standard structure, most prominently the fact that _no standard tool will work_ with the repo. Some examples: `ansible-test sanity`, `andebox`, `antsibull-nox`, to mention a few. It is absolutely a valid choice, if you want to have more control of the tools and processes around it, or if you have only a limited number of people using those tools. In our case, the idea/hope is for that collection to gather traction and build a larger community and as soon as one starts going that way, using standard tools makes life much easier to collaborators than using custom-made tools. Surely, the `molecule` testing in `community.openwrt` can be improved, but that is not the only type of tests we want to have. And having everything custom-made can limit the availability of people interested in volunteer time to the collection . So, I appreciate your input on this, and I appreciate the improvements suggested, but I think we should keep the collection compatible with the regular toolbox. ---
**Author:** ![kks](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/kks/32/10352_2.png) [@kks](https://forum.ansible.com/u/kks)\ **Post date:** [December 27, 2025, 12:36pm UTC](https://forum.ansible.com/t/integrating-ansible-test-integration-no-pun-intended-and-molecule/45018/7 "2025-12-27T12:36:06Z")
> [@russoz](#): > > _no standard tool will work_ with the repo All the standard tools will work for the collection inside the repo. You just have to run them inside collection folder. Should they work for repo or collection? I bet for collection. Ansible-test is actually works better you do not need to checkout repo in special folder ---
**Author:** ![russoz](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/russoz/32/127_2.png) [@russoz](https://forum.ansible.com/u/russoz)\ **Post date:** [December 28, 2025, 12:03am UTC](https://forum.ansible.com/t/integrating-ansible-test-integration-no-pun-intended-and-molecule/45018/8 "2025-12-28T00:03:28Z")
Sorry, you are right: they will work inside that directory. What I meant was that most collection projects out there will have this structure in which the tools run from the root of the project - many developers will have that `ansible_collections/community(or the applicable namespace)/` structure and cloned the git repository inside that. And they won’t be able to use the tools in the usual way they do for the other collections, myself included. That being said, I appreciate your commenting, and I know that sometimes it does make sense to go custom, but I don’t see a compelling reason for us to do that in the `communtiy.openwrt` collection right now. If that changes in the future, I will know exactly where to look for alternatives! 🙂 Thank you!