Executor Node Failover Behavior

[Sakshi]

Hi Team, Could you please provide your solution for the below three pointers:

• How can I configure AWX VM Executor nodes in a High Availability (HA) Cluster Load Balancer setup to ensure continuous availability and fault tolerance?
• What is the functionality of the AWX Instance Peer in AWX platform and how can we interlinked with the two executor nodes at a time into the AWX Instance Peer.
• What are all the main components monitoring should be enabled on the AWX VM executor node in order to not to receive any escalations from the SLA agreement with the stake holders.

[Sakshi]

Hi Team, could you please help us and provide the insights for the above three pointer queries on the AWX VM Executor Node, and the solution would be helpful for us to leverage the Ansible Automation on the AWX VM Executor Node without any hassles.

[Denney-tech (Caleb Denney)]

@Sakshi

• How can I configure AWX VM Executor nodes in a High Availability (HA) Cluster Load Balancer setup to ensure continuous availability and fault tolerance?

When you create Execution Nodes (EN), they will be added to an Instance Group (IG) in AWX. You can have as many IG’s as you like, but each EN can only be associated to a single IG. When AWX runs Jobs, they get placed into a queue, and the job gets assigned to an Execution/Hybrid/Control node in an available IG. The IG’s also keep track of the health state of each node, and only schedules jobs against healthy nodes. This provides you with High Availability through AWX, there’s no need for you to do any sort of HAProxy load balancing.

@Sakshi

• What is the functionality of the AWX Instance Peer in AWX platform and how can we interlinked with the two executor nodes at a time into the AWX Instance Peer.

IIRC, peering is for pairing EN’s to Hop nodes. Hop nodes act as Bastions or Proxy hosts between AWX and EN’s in isolated networks

@Sakshi

• What are all the main components monitoring should be enabled on the AWX VM executor node in order to not to receive any escalations from the SLA agreement with the stake holders.

AWX monitors the EN’s itself, but if you want to ingest into third party monitoring, there’s probably a way to check the health of your AWX nodes. You could see if https://awx.company.corp/api/v2/metrics/ has what you need, or if you specifically need each instance’s health state, you can go to https://awx.company.corp/api/v2/instances/ and iterate over the "results": [*].node_state to see if they’re in a ready state.

[Sakshi]

Hello,

Thank you for taking the time to reply. Could you please provide any supporting documents or URLs for our reference. Your assistance would be greatly appreciated. Thank you.

[Denney-tech (Caleb Denney)]

I generally use the AWX-Operator GitHub docs or the actual AWX GitHub docs, but there is also the readthedocs.io documentation.

• You can read about instance Groups here: 23. Instance Groups — Ansible AWX community documentation
• as well as Container Groups here: 7. Container and Instance Groups — Ansible AWX community documentation
• This goes over Mesh Topology, which shows hop nodes used as peers: 8. Managing Capacity With Instances — Ansible AWX community documentation
• You’ll probably be interested in performance (monitoring is mentioned here): 13. Improving AWX Performance — Ansible AWX community documentation
• and last but not least: 12. Metrics — Ansible AWX community documentation

There’s plenty of RTM material.