They’re both part of the Ansible Community Package. What should we do with those collections?
I think there are three possibilities:
Replace the old with the new collections (not a good idea because this would be breaking).
Add the new collections and remove the old ones in a later ACP major release.
Deprecate and remove the collections completely.
I tend to 3, and to do it ASAP. Maybe even deprecate them in 14 and remove them from 15. Especially since they didn’t do a new major release. Renaming a minor release and not doing a new major release for this violates SemVer IMHO.
If they are both in the package, and the change is a rename only, I don’t think it would be fair to make them resubmit to get into the package again. So I would say 2 (for whatever that is worth )
Edit: I do see now your point about this being a minor release. I think they’re communication or planning with ACP could’ve been better, but they did the name change is a reasonable way IMO
But at the end of the day, pinning and keeping those collections indefinitely isn’t a solution. I think we need to do something about this. I guess we’ll do either 2 or 3.
On the one hand, option 2 would be better for users. On the other hand, this means that we allow collections that are part of the ACP to do stupid, non-SemVer changes. And I don’t like this… actually, this means we’re working around their non-SemVer changes.
As I’ve said, I can live with option 2. I’m open to both options. Let’s see what others have to say about this.
The releases violated the inclusion requirements by adding a new dependency and removing content in a new minor release (without making sure the dependency is added ot the package first). This is a breaking change (for ACP, not for the collection itself since it adds the dependency) and not acceptable.
There is an established process of renaming a collection that’s part of the Ansible community package (namely Ansible Package Release Management), which wasn’t followed here. It explicitly asks for a new major release.
There’s also another problem: since the repository has been renamed/moved to another organization, both the original collection and the renamed collection now use the same repository and the same version numbers. This makes it impossible to satisfy the inclusion requirements that collections need to be tagged.
For that reason, option 2 is technically not possible right now. Option 1 would be possible, but would be very unfriendly to our users.
Sounds like you prefer option 3, like me. I’ll wait a couple of days to give people time to comment, and then start a vote on deprecating and removing those collections. Possibly withe the 4 options:
Replace the old with the new collections
Add the new collections and remove the old ones in a later ACP major release
I understand the reasoning behind why number 2 is technically not possible. I also (now) understand that the collection has violated some process/rules ACP has, which are in place to prevent a poor user experience.
As a maintainer of an ACP collection, I feel it is…difficult to both keep up with and know how to do everything that is required for membership. I am fortunate enough to have Mario look over my shoulder and stop me from doing something that might violate the rules. If I was on my own (and certainly if I worked for a company whose primary goal was not to publish and maintain Ansible collections) I think I would have made a few mistakes by now. I say that not to argue that the rules should be bent, but rather to suggest that reaching out to the collection/maintainers and telling them what is about to happen would be a nice gesture. The maintainers may even have some ideas on how the issue can be resolved.
As a user of Ansible, I see option 1 and 3 as equally disruptive. Unfortunately, that is the position the collection has walked itself into but I think its still worth calling out. If I use purestorage.flasharray today, it redirects fine. If you swap the old for the new, or remove the collection altogether, my workflow breaks. To be fair, I dont know whether option 1 or 3 allows for better communication of the upcoming disruption.
Not really. I don’t like any of the three options very much (I mean, I don’t mind if the Ansible package shrinks, but this isn’t really the way its size should be reduced.)
What we could do (but that requires cooperation of the collection maintainers) is:
a) Create new repositories for the “old” collection, that contain their sources and their git tags.
b) Use these repos for the old collections included in Ansible 15, and use the old repos (with the new name) for the new collections which are then also included in Ansible 15.
That way we have proper redirects and the new collections, and due to new repos for the old collections the technical issues for 2 go away.
I agree, it’s definitely not easy to keep up with all the rules. Maybe we should have a subset of these rules that describe the most important procedural things. IMO these probably are:
Stick to semver; in particular: a) backwards compatibility breaking changes only in new major releases, b) new features only in minor or major releases.
Make sure your CI is running (and usually passing!) and is testing the latest ansible-core versions (and all others that you claim to support).
Have a proper release workflow so that your releases don’t contain files they shouldn’t, and that releases are tagged in the repository. (All collections published through Zuul satisfy this automatically.)
If you use or want to add collection dependencies, check out the more detailled rules! In short: only add new dependencies which are already part of ACP, and make sure that lower and upper bounds on versions do not cause problems for ACP.
If the Ansible release managers create issues in your repository, don’t ignore them.
I think this covers almost all (or even all) problems we had in the last six years with collections already included in ACP.
This is usually what we do (and often helps to resolves issues). In the case of the two purestorage collections here, this didn’t happen; this is likely my fault. I created PRs to pin the collection versions (flashblade, flasharray) and mentioned the first one of them in the tracker issue. I did mention it in the release manager room on Matrix, though, and looking at the dicussion again right now, I see that something did happen on the forum: Namespace: everpure
In fact, before the rename, when the new namespace everpure was requested @mariolenz explicitly informed the maintainer that there is a procedure for renaming, and that we can help with questions. That happened before the new releases breaking the ACP build were published (which got me to pin the versions). There was never any reply to this message.
Looking at that thread again (I’m surprised that you cannot find the thread on the fourm when explicitly searching for everpure, which I did earlier today and now), it looks like the collections have no interest in staying in ACP.
In any case, pinging @sdodsley, in case there was a misunderstanding / …?
We specifically reached out the the Ansible Partners team at Red Hat to get advise on the appropriate way to rename the namespace, caused by our corporate rename.
We followed the instructions we were given, so I fail to understand how you can be threatening us with removal from ACP.
We also researched the rename procedure through all Ansible forums and documentation, and nowhere did we find anything that stated we were doing anything incorrectly.
We have full red Hat certification for these collections, so I would ask that you provide so detailed clarification on what specifically we did wrong, where this is documented and, more importantly, how we can resolve this matter to everyone’s satidafaction.
Creating a new major release now doesn’t make sense, since the breaking changes (from ACP point of view) already happened. The main problem now is that both the old and new collections are in the same repository and share the same version numbers, so you cannot tag the releases (which is a hard requirement for the build process).
The best way to resolve this would be as I sketched out above:
The new repos only need to contain the state of the old collection for these releases (and them being tagged).
So:
A new repository for purestorage.flasharray that contains the sources and tag for 1.45.0 (and potentially later releases, like 2.0.0);
A new repository for purestorage.flashblade that contains the sources and tag for 1.28.0 (and potentially later releases, like 2.0.0).
We explicitly pointed out the procedure to you in Namespace: everpure - #3 by mariolenz and asked whether you want to keep the renamed collections in ACP. Since you never replied and didn’t follow the linked procedure, we assumed you are not interested in having the renamed collections in ACP.
@felixfontein I didn’t seem to get a notification aboutthe second response to my namespace request, otherwise we would defintley have followed those instructions.
So the outcome we would like is that the purestorage collections (flasharray and flashablade) are replaced by the everpure namespace versions.
For backwards compatability, the latest purestorage collections of the same version numbers should be kept for one GA release - they are just stubs that install the everpure equivalents and provide deprecation warnings - then they can be removed.
If the best way to do this is crete a v2.0 of the everpure collections we can do that, but it will mean that the purestorage collections will need to be updated as well, to remove the dependency upper limit.
I would agree that there’s not really a good way out the situation. But just to get this clear: I want to remove purestorage.flasharray and purestorage.flashblade from the Ansible Community Package. I think this is important: They’re dead, there won’t be any fixes anymore. Especially not for incompatibilities with future ansible-core releases.
Creating new repos for the old collections would be a possible solution. But I’m not sure if this is really feasible.
How about adding everpure.flasharray and everpure.flashblade to ACP 15, and also purestorage.flasharray 1.45.0 and purestorage.flashblade 1.28.0? This way, we would have the redirects and the new collections they’re pointing to. This would somehow be against stop including new collections in the Ansible community package. But we could argue that the everpure collections aren’t new, just successors to already existing ones. And while it’s not great that the redirects and renaming has been done in a minor release, at least we won’t have this in a minor ACP release but in the next major release.
I didn’t have closer look at the new collections recently, so I can’t say if there are any issues that might prevent us from adding them to the ACP. Just thinking loud loud about what to do to get out of this situation.
True, but the right way forward is to replace them with ‘redirect’ collections (while also adding the new collections) so that users get deprecated redirects and find out that they have been renamed. The redirect collections can then be removed after some time.
That’s what I would do, but it requires new repositories for purestorage.* because right now, the same version is used by both the old and new collection in the same repo, so the version tag in the repo can only reference one and not the other.
I’d also add the everpure.* collections to Ansible 14 so that users who have to use Ansible 14 and 15 in parallel can already switch to the new collection names without breaking their playbooks/roles for Ansible 14. (This is what we did for other collection renames in the past as well.)
Oh, I didn’t spot this. I thought that they at least released the everpure collections with a new version. But you’re right, it looks like they’ve released the renamed collections with the same version, and therefor we have this tagging issue.
I think there’s no way out of this without new repos for the old collections. I guess they could be archived right away, but we need a unique repo + tag combination. Is this correct?
What about if we release new Collections for everpure.flasharray and everpure.flashblade. That would increment the version number. We are close to doing this anyway.
I thought that I could clone the collections, revert everything to the time when the namespace hasn’t been changed yet, and we could use this. But I don’t understand what has been released in the old namespace / under the old FQCN:
)
(I mean, I don’t mind if the Ansible package shrinks, but this isn’t really the way its size should be reduced.)