Run a command only on hosts that have low CPU usage

Archives Ansible Project
1

Hi Folks,
I am looking for a simple way to have Ansible perform the following automation for me:

  1. From a set of N hosts, find out which host has the lowest CPU usage at the moment
  2. Run a command on only that host.
    This pattern could be extended to a general case of:

Run a command on a subset of hosts matching a condition which is not static.

This could apply for the following types of use cases:

  • Find out which hosts are running low on disk space, and only expand disk space on the hosts which are running low
  • Find out which hosts are lacking a particular upgrade, and only upgrade those hosts
  • Only run a command on machines which are not currently under heavy load
    This seems like a common and desirable enough pattern, but as far as I can tell there is not a well defined way to execute this with Ansible. Is there a simple way to do this with a playbook, or do I need to write a strategy plugin?

Thanks,
Eric

2

Hi Folks,
I am looking for a simple way to have Ansible perform the following automation for me:

1. From a set of N hosts, find out which host has the lowest CPU usage at the moment
2. Run a command on *only that host*.

This pattern could be extended to a general case of:

    Run a command on a subset of hosts matching a condition which is not static.

This could apply for the following types of use cases:

  * Find out which hosts are running low on disk space, and only expand disk space on the hosts which are running low
  * Find out which hosts are lacking a particular upgrade, and only upgrade those hosts
  * Only run a command on machines which are not currently under heavy load

This seems like a common and desirable enough pattern, but as far as I can tell there is not a well defined way to
execute this with Ansible. Is there a simple way to do this with a playbook, or do I need to write a strategy plugin?

Thanks,
Eric

Hello Eric,

you could try to add the hosts to a new group with the "add_host" module and execute the command with delegate_to in
a loop over the hosts in that group.

Regards
         Racke

3

Racke,
Thanks for your response.

In practice, what would this look like?

So far, I have a playbook that I can use to get a list of CPU values. However, I am having trouble correlating the value of CPU to its corresponding hostname.

4

Racke,
Thanks for your response.

In practice, what would this look like?

So far, I have a playbook that I can use to get a list of CPU values. However, I am having trouble correlating the value
of CPU to its corresponding hostname.

---
- hosts: host_list
gather_facts: false

tasks:
- name: Get CPU usage
shell: "top -b -n 1 | head -n3 | tail -n1 | awk '{print $2}'"
register: top

- name: Set CPU fact
set_fact:
cpu_list: "{{ ansible_play_hosts | map('extract', hostvars, 'top') | map(attribute='stdout') | list }}"

Is this on the right track, or do I want a different approach entirely?

Thanks,
Eric

Hello Eric,

this can be done, but I doubt that it makes sense to do it with Ansible as above (especially with a lot of hosts).

Collect system metrics on each server with something like Telegraf and base your decisions on that.

Regards
       Racke

5

Create list of dictionaries

  - name: Set CPU fact
    set_fact:
      cpu_list: "{{ ansible_play_hosts|
                     map('extract', hostvars)|
                     list>
                     json_query('.{host: inventory_hostname,
                                     cpu: top.stdout}') }}"
    run_once: true

Find the minimal load

  - set_fact:
      cpu_min: "{{ cpu_list|json_query('.cpu')|min }}"
    run_once: true

and find the host with this minimal load

  - set_fact:
      host_min: "{{ cpu_list|json_query(query)|first }}"
    vars:
      query: "[?cpu == '{{ cpu_min }}'].host"
    run_once: true

HTH,

  -vlado