Howto execute ansible one host group at a time?

Get Help
1

hi,

I have a problem with Ansible 2.16.3(and can’t perform upgrade to newer version in an easy way) I didn’t expect to face.

I have playbooks I need to execute on multiple host groups but only one host group(HG) at a time(dependencies between services).

  • Hosts in HG2(50 hosts) can be executed after all hosts in HG1(20) finished.
  • Hosts in HG3(600 hosts) can be executed after all hosts in HG2(50 hosts) finished.
  • etc till Hex

I have ANSIBLE_FORKS set to 300 (infra limit).
It would be good keep it due to total number of hosts that must be handled.

Problem:
ANSIBLE_FORKS allows execution of HG1 and HG2 and part of HG3 in parallel to match total 300 and keeps rolling.

Things explored:

  • keyword serial but see no option to limit to “one HG at a time up to limit ANSIBLE_FORKS”.
  • –limit CLI parameter - doesn’t seem to work.
  • keyword delegate - works for one host only
  • import_task but by the time this is evaluated it’s already running on multiple HGs.
  • execution strategies available in default installation
  • host group parameter for play using “HG1:HG2:HG3” and “HG1 HG2 HG3”. In documentation I see no mention of “ansible waits for one HG to finish before next one starts”

Things I’d like to avoid:

  • I have a bash script to control Ansible execution. I’d like to avoid running Ansible multiple times in a for loop one HG at a time. It’s stupid to control execution strategy of Ansible from Bash script :-/
  • Writing a ton of short plays in one file for every single HG.
  • using serial and listing it like snippet below. We are talking 20HGs and bunch of different ansible plays.
 strategy: linear
  hosts:
    - group1
    - group2
    - group3
  serial:
    - "{{ groups['group1'] | count }}"
    - "{{ groups['group2'] | count }}"
    - "{{ groups['group3'] | count }}"

Things I’m thinking about:

  • could tags solve this?
  • could a custom execution strategy solve this? Is it available on Ansible galaxy or somewhere else? Can someone draft it if it must be written?
  • any other ideas?

Can someone suggest a solution?

What am I missing?

I can’t believe it I can’t find an explicit way to tell ansible “execute next host group once first finished”

thank you, Honza

2

You can have several plays inside playbook and this is something that might be helful in this case.

This is what comes to mind.

---
- name: Run on group 1
  ansible.builtin.import_playbook: playbook.yml
  vars:
    hosts: group1
- name: Run on group 2
  ansible.builtin.import_playbook: playbook.yml
  vars:
    hosts: group1

---
# playbook.yml
- name: Run tasks
  hosts: "{{ hosts }}"
  tasks: 
    ....

Did not try this to be honest.

1 Like
3

Q: “How to execute Ansible one host group at a time ?”

A: Put the code into a role(s) and import them conditionally. For example, given the trivial role test

shell > cat roles/test/tasks/main.yml 

- debug:
    msg: "Running at {{ inventory_hostname }}"

, the inventory

[HG1]
host_10
host_11

[HG2]
host_20
host_21

[HG3]
host_30
host_31

, and the playbook

- name: Import roles conditionally.
  hosts: all

  tasks:

    - import_role:
        name: test
      when: inventory_hostname in groups.HG1

    - import_role:
        name: test
      when: inventory_hostname in groups.HG2

    - import_role:
        name: test
      when: inventory_hostname in groups.HG3

The play runs as expected

PLAY [Import roles conditionally.] **********************************************************************************

TASK [test : debug] *************************************************************************************************
ok: [host_10] => 
    msg: Running at host_10
ok: [host_11] => 
    msg: Running at host_11

TASK [test : debug] *************************************************************************************************
ok: [host_20] => 
    msg: Running at host_20
ok: [host_21] => 
    msg: Running at host_21

TASK [test : debug] *************************************************************************************************
ok: [host_30] => 
    msg: Running at host_30
ok: [host_31] => 
    msg: Running at host_31

PLAY RECAP **********************************************************************************************************
host_10                    : ok=1    changed=0    unreachable=0    failed=0    skipped=2    rescued=0    ignored=0   
host_11                    : ok=1    changed=0    unreachable=0    failed=0    skipped=2    rescued=0    ignored=0   
host_20                    : ok=1    changed=0    unreachable=0    failed=0    skipped=2    rescued=0    ignored=0   
host_21                    : ok=1    changed=0    unreachable=0    failed=0    skipped=2    rescued=0    ignored=0   
host_30                    : ok=1    changed=0    unreachable=0    failed=0    skipped=2    rescued=0    ignored=0   
host_31                    : ok=1    changed=0    unreachable=0    failed=0    skipped=2    rescued=0    ignored=0

Q: “Would loop work?”

A: Yes, it would. But, in the loop, you have to use include_role instead of the import_role. See the difference between the static and dynamic use of the roles. Create the file include_role.yml

- include_role:
    name: test
  when: inventory_hostname in groups[item]

and iterate the groups’ list

- name: Include roles conditionally.
  hosts: all
  
  tasks:

    - include_tasks: include_role.yml
      loop: [HG1, HG2, HG3]

The play runs as expected

PLAY [Include roles conditionally.] *********************************************************************************

TASK [include_tasks] ************************************************************************************************
included: /export/scratch/vbotka/vbotka.ansible_qa/tasks/so-79874782/include_role.yml for host_10, host_11, host_20, host_21, host_30, host_31 => (item=HG1)
included: /export/scratch/vbotka/vbotka.ansible_qa/tasks/so-79874782/include_role.yml for host_10, host_11, host_20, host_21, host_30, host_31 => (item=HG2)
included: /export/scratch/vbotka/vbotka.ansible_qa/tasks/so-79874782/include_role.yml for host_10, host_11, host_20, host_21, host_30, host_31 => (item=HG3)

TASK [include_role : test] ******************************************************************************************
included: test for host_10, host_11

TASK [test : debug] *************************************************************************************************
ok: [host_10] => 
    msg: Running at host_10
ok: [host_11] => 
    msg: Running at host_11

TASK [include_role : test] ******************************************************************************************
included: test for host_20, host_21

TASK [test : debug] *************************************************************************************************
ok: [host_20] => 
    msg: Running at host_20
ok: [host_21] => 
    msg: Running at host_21

TASK [include_role : test] ******************************************************************************************
included: test for host_30, host_31

TASK [test : debug] *************************************************************************************************
ok: [host_31] => 
    msg: Running at host_31
ok: [host_30] => 
    msg: Running at host_30

PLAY RECAP **********************************************************************************************************
host_10                    : ok=5    changed=0    unreachable=0    failed=0    skipped=2    rescued=0    ignored=0   
host_11                    : ok=5    changed=0    unreachable=0    failed=0    skipped=2    rescued=0    ignored=0   
host_20                    : ok=5    changed=0    unreachable=0    failed=0    skipped=2    rescued=0    ignored=0   
host_21                    : ok=5    changed=0    unreachable=0    failed=0    skipped=2    rescued=0    ignored=0   
host_30                    : ok=5    changed=0    unreachable=0    failed=0    skipped=2    rescued=0    ignored=0   
host_31                    : ok=5    changed=0    unreachable=0    failed=0    skipped=2    rescued=0    ignored=0

source code

1 Like