Thanks @Jeff_Pullen, @kks and @chris for your replies in this thread. I want to provide my thoughts on the areas raised above.
Scope: A dedicated post to split the role only discussion
This post was to focus specifically on collection testing. Role only testing is a critical part of the Ansible content ecosystem, but it is a different problem with a different persona, YAML first rather than Python first. Mixing the two into one discussion makes the discussions in this post about collections more confusing.
I’d like to propose that we move the role testing replies above into a separate post, dedicated to that topic. This will help ensure collection testing decisions do not taint role only concerns and vice versa.
Molecule: Playbook copying between scenarios
On the point about copying playbooks across scenarios, you don’t need too. A common playbook can live once and be imported into each scenario with the ansible.builtin.import_playbook module. Please see the examples below:
- Common playbook.
- Reused across scenarios: Example 1, Example 2.
The same goes for molecule’s shared_state: true. It’s not collection only, molecule/default at the root of a role repository works the same way as extensions/molecule/default inside a collection. extensions/molecule is a collection convention but not a hard requirement for this functionality. So the point in #37 that you end up in collection territory the moment you want supported testing is not strictly true IMO.
Lack of documentation is the causing factor for several issues
The more I look at these replies, the more I think a lot of this comes back to documentation, and the lack of it for role testing specifically. For example, the playbook copying problem above is likely a symptom of this documentation gap. The import_playbook pattern and shared_state in role mode are not documented anywhere (that I know of) currently, so when documented examples show a full copy of create/destroy per scenario, authors copy. When search returns the driver:/platforms: style, authors follow that.
So, I think a real action item here for us is to focus on documentation for Molecule and on docs.ansible.com too for role testing. I feel like this would go a long way and help a lot. What do you think?