There’s a lot to digest here, and I have a lot of ideas to share. I’m going to try and stay on topic here while I share my feedback.
While I don’t expect AWX to support “stable” release branches, I’ve always been annoyed that there’s no reference point for what AWX version/commit matches what Ansible Controller version downstream. Even with the understanding that AAP will apply changes to its AWX fork downstream, it would still be nice to know from which commit AWX was forked from. Creating an “aap-x.y” branch in AWX for subsequent AAP releases would be welcome. It might even be useful to your internal team if you needed to back-port fixes applied upstream instead of cherry picking them after the fact.
So there’s a few different things around “breaking changes” and release cycles I want to talk about.
Code deprecation/breaking design changes
These things should fit somehow into your normal release and announcement workflows. Whether or not deprecations should be given a specific support period before fully removing them is another topic.
Releases with unintended breaking changes
Things like the awx-operator deleting the wrong PVC or in the wrong order of operations for things like postgresql upgrades, or changes that otherwise prevent parts or all of AWX from working as expected. Particularly borked releases need a process for purging all related tagged images (not just the main culprit images, but things like operator bundle/catalogs/etc). As well as integrating with the regular announcement workflow to notify users of known issues, removal of these images, and whether or not to expect a hotifx or wait for the next release cadence.
CICD is a must
Having some level of CICD built around addressing breaking changes is definitely a must after you switch to CalVer. With SemVer, if x.y.0 introduces breaking changes, they may get fixed in x.y.1, so as a general rule, it’s safe for consumers (users) to assume that a given z version should be skipped if there’s any subsequent z version. Hotfixes could be released even the same day. I’m not sure how same-day-hotfixes would look in CalVer, but regardless of what that tag looks like, it might make sense to remove images with breaking changes altogether (whether that’s the main image, the operator, or the operator’s catalog/bundle/manfests/etc). If “bad” images are removed, there’s less concern (for us users) about using the ones that remain.
PS about awx-operator-catalog
Additionally, the awx-operator-catalog could be improved with more detailed entries, replaces, and skips/skipsRange that exclude the broken images, but only if that can easily be automated (I do this easily enough for SemVer, but CalVer might be a headache).
The operator catalog release cycle itself is something else to think about. As it is right now if you switch to weekly/nightly releases, every release would throw PR’s at the community catalogs, upstream and downstream. I don’t know how automated or noisy that might be.
YYYY.MM.DD for sure. Including leading 0’s as @eburgueno suggested.
Edit: I’m actually thinking maybe without leading 0’s to keep the CalVer consistent with SemVer enough to allow “version type” tooling to continue working as expected. Maybe even doing weekly release versions with incremental z version for out-of-cycle patches, i.e: YYYY.WW.0 YYYY.WW.1
Make posts to all the usual release announcement mediums notifying the breaking changes, presenting any disaster-recovery processes if applicable (like that PVC deletion one a few months ago), and purging those particularly broken images from Quay.
Maybe at most a weekly release cadence (Mondays? So hotfixes can be applied quickly during the work week if needed), and a separate nightly tag for nightly builds if they are desired.