Hi,
I have deployed several Execution Nodes which are all 4 cpus and 16GB but they all show as 0.5 cpu and 2GB in terms of capacity in AWX, which seriously limits their workload!
How can we allocate the full host’s specs (4cpu/16ram) to the exec. node so that it properly reports back the available capacity to the Control Planes?
I was able to adjust on the task container specs on the Kubernetes side (even tho rsyslog_resources still creates an error on my end) but still cannot fix the exec. nodes…
Thanks!
That’s Container not your host machine?
Execution nodes are host machines (running jobs using ansible-runner and podman) completely outside (and far from) the Kubernetes cluster (AWX control planes).
My issue is they don’t report their fulls specs to AWX control planes (reporting 0.5 cpu instead of 4 for example), which limits their usage… I wonder if it’s how podman is run by ansible-runner or something else in the receptor/ansible-runner?
JC
AWX is getting those cpu/mem measurements by running
ansible-runner worker --worker-info
you can ssh into those machines, change to awx user, and run that command above. If you want to look deeper into those who values are obtained, here is the source code
https://github.com/ansible/ansible-runner/blob/0746379fbf31c357a75946183b23dda06ea777f8/src/ansible_runner/utils/capacity.py#L8
Let us know if that helps a bit
AWX Team
Hey, I found the problem!
I did check ansible-runner infos and it was showing the right specs for the exec. nodes matching the hardware.
BUT, after testing and confirming, the resources requirements that are set for the k8s task container are ALSO applied to the podman containers running on the exec. nodes!
I edited the deployment’s resources and applied, after a new health check the exec. nodes now show the new limitations.
I think it’s not best because the exec. nodes can potentially use much more resources than what we want to allocate/limit on our awx_task container(s) for the Control Planes.
It would be great to ensure that podman on the exec. node side doesn’t get limited by awx_task container’s resources limits, or that exec. nodes get their own limitations.
Thanks!
oh this is an interesting finding and is a bug I believe. I will put up a bug report
AWX Team