Your Feedback Wanted: The Ansible Collection Developer and Maintainer Journey

Project Discussions Collection Development
, , , , , , ,
1

The Ansible community and partner engineering team at Red Hat have recently gone through the collection developer workflow using Ansible Development Tools.

The purpose of that activity was to gain a shared understanding of the tasks and challenges that developers and maintainers face when creating, testing, releasing, and maintaining Ansible collections.

As an outcome of that activity, we have collaboratively described the main milestones and high-level steps involved in the collection developer workflow. These milestones outline the journey that developers and maintainers go through to succeed in their efforts, whatever their motivation may be.

We see this journey as common across all collections, whether in the Ansible community or in Red Hat Automation Hub downstream. We want to see the results and feedback to be used as context for upcoming work in ADT teams, Community and partner engineering team, and the community to improve the collection developer experience.

We would love to hear your feedback and invite you to share your thoughts and opinions. We’re also trying to reveal and understand obstacles, pain points, and challenges that collection developers and maintainers face. In particular:

  • Are there any milestones or steps within the milestones we missed?
  • Should anything be updated or removed as non-essential?
  • What challenges have you faced during collection development and maintenance?
  • How can the journey be used to benefit the community?
  • If you have additional feedback on any of the milestones and steps below or on this topic in general, please let us know.

Create a new collection milestone:

  • Initialize and create a new source repository
  • Initialize a collection structure on the filesystem
  • Familiarize with and update the essential files like README.md and galaxy.yml
  • Apply the appropriate open-source license

Normalize development environment milestone:

  • Create a devcontainer and/or devfile
  • Add pre-commit hooks to source control to enforce consistent style and formatting
  • Understand the different types of tests: integration, unit, sanity, linting
  • Choose and configure a testing framework
  • Set up a basic CI pipeline structure
  • Ensure CI runs the appropriate level of tests
  • Configure CI checks for Ansible Lint
  • Adopt a versioning convention like SemVer and clarify the types of changes (major, minor, patch)
  • Install antsibull-changelog and configure changelogs

Add automation content milestone:

  • Write tests alongside roles, modules, and plugins
  • Create roles to organize playbooks
  • Create modules and plugins
  • Document automation content in user guides
  • Specify collection metadata
  • Add dependencies and requirements to the collection in files such as galaxy.yml, bindep.txt, requirements.txt, and README.md
  • Add supported Ansible core version to meta/runtime.yml

Automate the release pipeline milestone:

  • Create an execution environment definition
  • Configure CI check for Galaxy Importer
  • Add CI jobs to release to Galaxy and/or Automation Hub
  • Add CI jobs to build EEs and push to container registry

Maintain collection lifecycle milestone:

  • Establish contribution guidelines (CONTRIBUTING.md)
  • Set up a process for regular dependency updates, bug fixes, and security patching
  • Adjust test matrix for Ansible core and Python versions according to core lifecycles
  • Update meta/runtime.yml with core version to reflect changes
  • Evaluate and integrate new types of automation content
  • Review and adjust for changes in collection inclusion and certification requirements
  • Document collection design and architecture for future maintainers

We’d love to hear from you in the comments! :wink:

2

There’s a ton of stuff in there that’s nonessential. Looking just at the “Normalize development environment” milestone, we have:

None of that is necessary to do collection development.

And this one raises the question of who the target audience is:

SemVer is a hard requirement for collections intended to be part of the community distribution. People who aren’t interested in that could adopt a different versioning scheme, but they could also ignore even more of the things listed, like doing CI at all.

It’s also weird to me to split this up into milestones that impose arbitrary ordering onto a more fluid process. It’s particularly strange in the scattering of various metadata updates among them. Like, first I’m “updat[ing] the essential files like galaxy.yml”, then two milestones later I’m “specify[ing] collection metadata”, which is vague but I guess doesn’t include adding a license, updating the metadata in meta/runtime.yml, or specifying requirements because those are separate steps? I’m not sure what that leaves that I wouldn’t have done the first time I edited galaxy.yml, apart from (optionally, for most people) defining an execution environment…which is in the next milestone for unclear reasons.