Proxmoxer is a great library that allows you to access the proxmox rest API, but that requires to make a remote access request for each task, which among other things requires authentication management
Given the fact that ansible tasks are already running directly on the target machine, what’s the reason for these modules to not just directly manage the proxmox instance locally either by modifying the configuration files directly or by calling the proxmox cli?
This is out of curiosity
Hi there 
,
Once upon a time someone decided to make modules for proxmox and they decided to use proxmoxer in order to abstract the API and not implement that themselves.
I wasn’t involved in the development or the decision to do so.
From the practical side of things, it’s mainly about flexibility, running tasks in proxmoxer can be done on the Proxmox host itself, however, it doesn’t need to be.
If you’re running tasks on a PVE server on which you don’t have shell access for example, with proxmoxer you still can run automation against your own VMs (or, in the situation where you can also make VMs, still create those).
And not all modules are based on proxmoxer, I wrote the initial version of the community.proxmox.proxmox inventory and it triggered the same discussion (https://github.com/ansible-collections/community.general/pull/545 and add inventory plugin for Proxmox VE by xoxys · Pull Request #736 · ansible-collections/community.general · GitHub).
Back then I was a bit skeptical about using proxmoxer myself (‘because it was just an abstraction of an abstraction’, silly me) and I wanted to be able to only inventory VMs on systems on which getting proxmoxer was a challenge on it’s own. And it seems that the author of the other PR agreed, as that one was closed in favor of the one I wrote (but still many thanks to the foreman guys for making the framework I ripped from them 
)
So, TL;DR: reasons, but it can be a bit more flexible when using them, especially when you do not have shell access to the PVE node(s).
I didn’t work on the module, but I can hazard a guess.
The cli tools are pretty powerful, and you can easily brick your instances, and probably the server as well if you get carried away or misunderstand an option. The API on the other hand takes care of a lot of stuff like permissions and sequencing of requests, in such a way that its somewhat more difficult to break things. The API is indeed how the developers want you to interact with the service, and has had a lot of testing with various client libraries.
The API however, isn’t perfect and has rough edges for historical reasons. The developers of proxmoxer have done a lot of work to cover the API features sufficiently for most purposes, and the community has tested that integration in the real world.
So basically I think it would take a lot of work and careful consideration, if you were to build a proxmox module based around the CLI, that is already available between the API and proxmoxer.
That said, I have a bunch of ansible shell utilities for doing ad-hoc stuff directly with cli commands because I couldn’t do it with proxmoxer or the module.
Excuse my naivety, but this was confusing for me as as someone just starting out. I went through the docs and gained a basic understanding of Ansible as a way to automate the ssh connection to any number of hosts to run prebuilt python scripts on the target machine to ensure a machine was in the desired state. From there I setup my proxmox VE host and pre-authed my ssh keys for easy automation. It was then confusing why I would need the proxmox user and password to re-auth into a machine that I was already running on. After some digging in the code itself I landed on the usage of proxmoxer as a REST api to call back into the target proxmox host.
So effectively I have: [local os] → [proxmox pve1 (Ansible)] → [proxmox pve1 (REST)]
This is redundant, confusing and seems tangental to the goals of Ansible. If I wanted to use the REST api to edit my proxmox cluster I could just run the script locally.
Hi there,
I think you have some fundamentals with ‘Ansible targets’ mixed up 
The reason you (in the case you describe) need to provide multiple sets of credentials is very simple; You’re targeting 2 different instances/services:
You can compare it with having a Linux box on $CLOUDSERVICE: if you want to manage the VM itself, you need to log in to the service, but logging in on the service does not automatically grant you permissions on the VM’s host OS.
My tingling sense tells me there are some delegate_to: localhost missing for proxmox modules?
With delegate_to: localhost you should get:
[local os] → [proxmox pve1 (REST)]
After some more testing I realized I do have an extra layer that is not required. I did not realize you could target localhost to run things against your current machine. So I now have:
[Local OS] -> [Ansible LocalHost] -> [Proxmox REST API (using tokens or PAM)]
This removes the redundant auth in a single operation, however it does still require that I configure a secondary auth token so that I can use the REST api. I assume the REST API layer was created to allow external computers/services to make changes to a cluster they do not have direct access to (no ssh). However, Ansible already gives us direct access to the proxmox server that is running these services. Why not leverage these existing auth flow for connecting to the server and call the existing command line tools to make the changes?
Because that’s not what the person(s) who wrote the existing modules wanted to do. You can equally ask, why require setting up an auth flow that allows logging in to the host server and calling command line tools, when there’s a perfectly good API you could use instead?
If you want modules wrapping the CLI tools instead of calling the API, someone (you?) has to write those modules, they’re not going to just magically appear.
Yes, that is why I am asking. Was this an arbitrary decision that was made by the original author, or is there a major limitation that prevents CLI based proxmox management from working.
This is exactly the reason why I started this thread at some point. I can understand the appeal of just accessing the REST API provided by Proxmox, but everything that the REST API can do can also be done through the CLI, and even better when you have access to modify files directly from Python. Also, I’ve found that managing errors in the API is tricky at best, and there are a handful of race conditions that seem to be intrinsic to the way the API works that are very hard for the user to work around
I also understand that this is someone’s work who made a conscious decision to use that API instead of the CLI, as that was probably an easier lift to get a set of modules out the door. I do appreciate the work that the original authors have put onto this, and I benefit from them. Thanks
I’m not sure that I would have made the same decision, but as @flowerysong mentioned, I’m free to build my own set of modules for this. I have already done that for managing network bridges, and I will probably open source it at some point. If I do, I will probably not make it part of the community.proxmox. namespace, as the approach for them are strictly different (accessing the node instead of using the API for this). I’m looking forward to getting some of my modules out the door, but I’m also very tight with timelines, so not sure when I’ll have time for that
Well, that’s very easy to answer :-), the reasons not to use the CLI tools are:
proxmoxer on a specific host for whatever reason (Execution environments that are hosted by a enterprise IT team in AWX/AAP come to mind). Which means that you need to delegate the tasks to a host you can controlSo for me it’s very logical to use the REST API instead of the CLI tools, it’s more flexible.
And the reason to use proxmoxer instead of ‘just accessing the API’, well, that’s more nuanced. The biggest benefit is that we (the Ansible Community) don’t have to adapt to subtle or not so subtle changes to the REST API on the PVE side. Which means we can focus on developing functionality instead of fixing compatibility, as all the edgecases are handled by the library.
The downside of having to use proxmoxer is that you have to install the library. Which is the main reason behind having the inventory plugin not use it. As this makes the inventory usable in the above mentioned scenario’s.
You mentioned that you’re running into weird issues with the API, those should be reported to the PVE devs, as I’m pretty sure that if there is anyone that can fix those, it’s them.
That all said, if there’s a reason to have some modules use the CLI, I don’t see a reason to exclude them from the collection. But, because of all I wrote above, I probably wouldn’t accept PRs that replace modules with CLI equivalents, especially if they provide redundant functionality.
And at the end of the day, none of us are explicitly getting paid to maintain this, we mostly do this out of the goodness of our hearts (and I think because we’re nerds that love tech <3).
While I’m not involved in development of modules for Proxmox, I have experience in developing modules for another VM platform. Beside everything alread said by other people, my take on CLI vs API is practicality: