Help with async tasks.

Archives Ansible Project
1

Dear Tech Team,

I’m Guannan from China. I’m new to Ansible and looking for some technical help.
Actually I’m looking for a method to execute one role in parallel with loops of different configuration files. For example, I want to create 100 sites using role “create_site” with 100 different configuration files. I simply write a playbook like this:

  • hosts: localhost
    gather_facts: no
    tasks:

  • name: register files
    shell: “find /tmp/site_files -name ‘*.yml’ | sort”
    register: file_names

  • include_role:
    name: create_site
    vars_from: “{{ item }}”
    with_items: “{{ file_names.stdout_lines }}”

This will create 100 sites in serial. If create one site take 2 minutes, then the whole playbook will take over 200 minutes. I found ‘async’ option when execute tasks, however it seems only used for async jobs on different hosts. Do you guys have experience or methods on how to execute one role with different configuration files in parallel? Please send me E-mails.
Much thanks for your help!

2

Dear Tech Team,

I'm Guannan from China. I'm new to Ansible and looking for some technical help.
Actually I'm looking for a method to execute one role in parallel with loops of different configuration files. For
example, I want to create 100 sites using role "create_site" with 100 different configuration files. I simply write a
playbook like this:

- hosts: localhost
gather_facts: no
tasks:
- name: register files
shell: "find /tmp/site_files -name '*.yml' | sort"
register: file_names

\- include\_role:
    name: create\_site
    vars\_from: "\{\{ item \}\}"
  with\_items: "\{\{ file\_names\.stdout\_lines \}\}"

This will create 100 sites in serial. If create one site take 2 minutes, then the whole playbook will take over 200
minutes. I found 'async' option when execute tasks, however it seems only used for async jobs on different hosts. Do you
guys have experience or methods on how to execute one role with different configuration files in parallel? Please send
me E-mails.
Much thanks for your help!

What is the "create_site" role actually doing?

Regards
         Racke

3

Thank you for your response.
Well, in my condition, I using ‘create_site’ role to create a site in a netbox. It will create a site with name, region, and some other fields, associate some devices and vlans with the site.
You may just treat ‘create_site’ as a role execute some tasks in serial.

在2020年8月31日星期一 UTC+8 下午5:45:56brae...@gmail.com 写道：

4

Hi ! maybe something like this :
https://medium.com/developer-space/parallel-playbook-execution-in-ansible-30799ccda4e0

5

Thank you for your response.
I’ve tried to add ‘strategy: free’ in my playbook, however it seems not work as my expectation. I think it may because there is only single host in my scenario, while ‘strategy: free’ works well in scenario of multi hosts and multi tasks.
We’re now trying to run multi jobs at the same time on AWX. One jobs call the role ‘create_site’ and loop for 5 configuration files, so that 20 jobs will cover 100 configuration files. It seems work in my scenario.
Thank you anyway, and good-looking signiture :slight_smile:

在2020年9月1日星期二 UTC+8 上午4:10:14mar...@etcheverri.com 写道：

6

well other option (i used only once) is run the shell command with gnu parallel .
its pretty amazing! but i pretty stiff learning curve.
maybe you could , give it a chance.

best regards!

7

Thank you!
Even I have not used gnu parallel before, I’ll also take it into consideration.
If have any progress, I’d like to share.
Best reagards!

在2020年9月2日星期三 UTC+8 上午8:07:53mar...@etcheverri.com 写道：