Some thoughts from CfgMgmtCamp about collection testing tools

Hi everyone,

I can’t believe it’s already been over a month since CfgMgmtCamp. There were many great discussions and ideas exchanged. One of the gems, from my view, was the session @felixfontein gave on antsibull-nox project.

Felix’s talk was a real standout because I think it shows a way to move forward despite technical challenges. The very nature of open source, and maybe IT fundamentally, is to work around when things get stuck.

And, as a tool goes, antsibull-nox provides a really elegant solution to problems that seem to plague collection testing. In his session Felix covers how many of the existing tools for testing collections are brittle and not even specifically designed for collection testing.

So if you still haven’t watched it, I highly recommend checking out Felix’s session: How to use antsibull-nox to test your collection.

Another thing that came up at CfgMgmtCamp was why the Community engineering team at Red Hat and the Partner engineering team at Red Hat were combined. There was a lot of overlap between the two teams given the focus on automation content.

If you’re not familiar, the partner engineering team (now the community and partner engineering team) owns the process for certifying and validating collections from Red Hat partners. As you might have noticed from some other recent posts, we’ve been talking about how to bring those partners closer to the rest of the community to share expertise and provide more transparency. There are a lot of benefits for doing that. One of those happens to be solving the same sort of frustrations that Felix described so well and that antsibull-nox is well suited to.

antsibull-nox provides a common interface to multiple collection testing tools. It also offers a very convenient way to test your content. Just add a straightforward, human friendly TOML file to your project and a couple of dependencies.

For the CPE team, we see antsibull-nox as a great way to increase the quality of content while also making it easy for collection developers and maintainers, such as Red Hat partners, to get past some of the headaches and obstacles caused by all the wrangling with different test tools. In the specific case of partners, antsibull-nox also has the potential to accelerate onboarding for the certification process.

This actually came up in a conversation with @Leo recently. He has been doing a lot of work with developer tools and was wondering how antsibull-nox and tox-ansible relate to each other, which brings me to my point.

My team and I have done a bunch of work to reduce fragmentation across the Ansible ecosystem. While it’s great to have more tools and additional flexibility, it can be confusing for folks trying to figure out which tool they should use when presented with options that seemingly overlap. So I’ve been thinking that it might be a worthwhile discussion to have in the forum.

As part of the Ansible Development Tools (ADT) suite of packages, tox-ansible is a good choice when following the Red Hat opinionated path of content development. Maybe also some of the experts out there prefer ini file configuration or have technical reasons for using tox-ansible.

On the other hand if you have a collection that has already been around for some time; if you scaffolded a collection from the community template; if you took over collection maintenance after a team handover, then antsibull-nox provides a convenient option that you can basically drop into your project. It seems like a solid option for folks who are a domain expert in a given technology but not necessarily an expert in Ansible collection testing.

At the end of the day both projects are tools with different approaches to doing a thing. While choice is always good, and healthy, it’s useful to have enough context to make informed decisions. So I’m hoping to kick off a bit of a discussion about all this to hear what other people think. Does the above make sense for the most part? Does anyone disagree? Has anyone used either antsibull-nox and/or tox-ansible and switched from one to another?

Looking forward to hearing some other points of view. Cheers.

1 Like