The title of the previous topic slightly tickled me for some reason, so I couldn’t resist re-using it.
I’ve spent the past several weeks doing a deep dive (for a newbie) into ansible with the aim of creating some new modules to fill in some gaps in the community.general.proxmox_* modules (for myself, at least - I’m not sure what may be omitted intentionally). E.g. there is a module for getting storage configurations, but not setting them. I feel like I’m now in a good position knowledge-wise to begin, but I thought I’d touch base with the community first before getting too wrapped up in my own thing.
It seems like there was some interest a year ago in moving the community.general.proxmox_* modules to a community.proxmox collection (as mentioned in the post link in the first paragraph; I also found a community.proxmox repo with a single commit forking a template repo). I’m probably in a good position to attempt that (with a little guidance on expectations). Is there still some interest in that migration and/or interest in a volunteer?
I’m also wondering how much interest, if any, there might be in upstreaming potential future modules. Some of the missing functionality feels conspicuous (like it was intentionally decided to be omitted, rather than just is currently unimplemented), but that’s just my intuition and I don’t know if it’s accurate. Is there any interest in additional modules for the official collection?
The effort to move the proxmox modules/plugins to community.proxmox was discussed last year at CfgMgmtCamp, but the folks involved found out that it was more work than they expected / had time for, and thus the project basically died before it really took off. Since then nobody stepped forward to take this up again - until now you did, I guess
Right now there are a few folks who sometimes work on the proxmox modules/plugins in community.general, as you can see from the output of git log plugins/modules/proxmox* plugins/inventory/proxmox.py plugins/connection/proxmox_pct_remote.py. So there is definitely a user base around and some hands to help, at least with smaller tasks, maybe also with larger ones (I don’t want to promise other people’s free time ).
The aim should be to get the collection in a workable state with basic CI, and to get a first release out, and soon a slightly polished one as 1.0.0.
Then we can deprecate the content in community.general and add community.proxmox to the Ansible community package, and eventually remove the proxmox content from community.general 11.0.0 (to be released in June 2025), resp. replace it with redirects to community.proxmox.
Basically from that point on the main development will happen in community.proxmox. Bugfixes added there should be re-created for community.general (since the stable-10 and stable-9 branches should still get bugfixes for some time), while PRs in community.general that are not re-creations from community.proxmox will be closed right away.
I think the main expectations are:
Implement the move, and ping interested folks (like everyone who created a proxmox related PR in community.general recently, or also folks who created issues and look interested in helping, and of course the @proxmox-collection in the forum - which you should join, if you haven’t). I’d start with forking the community.proxmox repo and figuring out how to get the history there, and once it works the Ansible community team can help taking over the community.proxmox repo and implement it there as well. (These things usually take several tries to get right, so it’s best to not do it in the real repo right away.)
Help getting CI up and running, at least a basic one. Running integration tests would be especially awesome (community.general only runs unit and sanity tests right now for proxmox content), but that is probably quite some work. (That’s why I would probably not require this for a 1.0.0 release.)
Make sure that the 1.x.y releases are (mostly) backwards compatible (with explicitly documented differences, if something isn’t backwards compatible!) to the current content in community.general, to ease migration.
At least stay around for some while to help with getting fixes and improvements merged and new versions released.
I’m sure that there are enough folks around who can help with everything, but who might not have enough time to fully work and implement one (or more) of these points.
Thanks for the detailed response, that’s helpful. I did go ahead and join the proxmox forum group.
I think I’m up for the challenge. With regard to timing, I should mention that health issues make the frequency and duration of my availability erratic. That is to say: Where timing is critical, I’m probably not the right person for the job. I’m also a bit rusty working in a team environment.
If that’s not a deal-breaker, one of the lingering technical questions I had is what should be done with shared code. In glancing through code looking for an example, though, it actually seems like the coupling might be loose enough for that to be a non-issue. There’s tight coupling to internal_test_tools, but I’m assuming that dependency can/should remain.
Otherwise, I have subscribed to the issue with the pending merge and will keep an eye out for further communication.
Well, depends on how you understand critical Timing is somewhat critical, but there’s a lot of time. The part I cannot judge is how to extract the necessary changes from community.general’s history (to preserve history). I did similar things in the past, but I can’t remember how exactly git filter-branch worked. But it shouldn’t be that hard to figure out. Polishing the collection and setting up basic CI also isn’t that much work (I can also help with that). I don’t think it should take more than a working day or two, depending on how much you know about collections work.
(Regarding history: note that the structure of plugins/modules/ was changed at some point in the past; the modules were moved from plugins/modules/cloud/misc/proxmox* to plugins/modules/proxmox*. That’s probably the most tricky part, next to finding all current locations of proxmox files: plugins/*/proxmox*, tests/unit/plugins/*/*proxmox*, and tests/integration/targets/*proxmox* and its subdirectories.)
The only shared code that isn’t part of the community.general repository is some shared code in community.internal_test_tools that’s used for the unit tests. If you don’t want the external dependency, you can simply vendor the code.
(It was already present in community.general until recently (back from the ansible/ansible split-up), when I decided to stop repeating the code all over many collections, and use the shared copy from community.internal_test_tools - especially since parts of it needed a bigger modification for Data Tagging support. But it’s not that much code, so vendoring would also be fine. Whatever you prefer.)
In case anyone missed it, that one is merged by now.
I have started work on a small script that will create the new repo using git-filter-repo (git documentation is recommending that over filter-branch these days). The script has a function to follow renames through history and did find the plugins/modules/cloud/misc/ files.
I think the script (and the repo it produces) will be in a shareable state in the coming days. I will ping @felixfontein and @apple4ever on GitHub when I have something posted. I will probably have more concrete questions then, particularly if I haven’t figured out the shippable part of the test suite by then.
Good progress! I’ll try to make some time to chip in once you have the repo up and running. I’d be interested and willing to help in trying to set up some integration tests with a VM proxmox instance (as I think those are a lot easier to set up then unit tests and I think it’d be a fun project to work on )
).
and I think it’d be a fun project to work on