I am looking for running multiple commands on single target via loop.
However i noticed commands runs in serial manner instead of parallel.
Is it possible to make loop to run in parallel rather waiting for each loop to complete.
Exa:
creating 10 users on target server. Passing user list via file.
==========
tasks:
I believe what you are looking for is configuring “forks” - Difference between Fork and Serial in Ansible
Forks influence the number of hosts task will be applied to in parallel. Each of those parallel hosts will still apply task loop items in serial.
The short answer is you cant run loop items in parallel. While there is a way to technically do it (duplicated inventory hosts comes to mind), this is a conceptual problem.
user Ansible module uses/calls useradd command on the host to create users. This command cannot run in parallel because it has to atomically change some files on the file system like /etc/passwd and /etc/shadow. That being said, users can only be created one by one.
What are you hopping to gain by running task loop items in parallel?
You can do so with host aliases, but I do not recommend running in parallel tasks that update the same resource, it can create contention and race conditions.
my use case is where each task is taking 2 -3 minutes so instead of waiting that task just fire one more and so on until loop completed.
so it will reduce time for execution of playbook.
Each task is taking 2-3 minutes to finish or each loop item in this specific task is taking 2-3 minutes?
In either case, this is a sign of some other problem. This is too slow to be normal.
each loop item in this specific task is taking 2-3 to complete.
I am looking something we do in for loop where using & , command goes in background for that iteration and other loop items processed
And how much time does it take to run useradd command on your hosts manually?
You have some major issue here. Trying to run anything in parallel will probably not going to be any faster.
useradd was just example…I know useradd will be quick.
I am looking for parallelism of loop which triggers task in one go rather, each loop iteration goes in sequence…
I’ll give you what you ask for, but be mindful, that this does not solve your underlying problem (probably high latency/bad dns to your network auth system).
hosts.ini
[parallel]
hostalias1 ansible_host=host1
hostalias2 ansible_host=host1
hostalias3 ansible_host=host1
hostalias3 ansible_host=host1
play
hosts: parallel
tasks:
name: Create users
user:
name: "{{ user_list[groups['parallel'].index(inventory_hostname)].name }}"
createhome: yes
*updated to use the ‘list of hosts in parallel’ to select the existing user_list
This way you target multiple hosts in parallel (but the same host in the end), this can/will create concurrency problems and race conditions, but it is what you were asking for.
Hi Brian, above playbook is for running a task on multiple hosts.
Here is example of my playbook.
hosts: localhost
connection: local
gather_facts: false
tasks:
name: Register loop output as a variable
ansible.builtin.shell: “date +%s%N ;echo {{ item }}”
loop:
name: Print output
debug:
var: echo.results
Check for start time of each loop iteration. Since date & echo command takes less time you will see stdout_lines with millisecond difference.
Each loop iteration is takes place in sequence( rather after completion first one) that is what i am observing
Assume that if we replaces echo/date commands with commands/script which takes more than 3 min. so each iteration will take 3 min. If we have loop of 50 count.
$ ansible-playbook parallel.yaml
[WARNING]: No inventory was parsed, only implicit localhost is available
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match ‘all’
PLAY [localhost] ***************************************************************************************************************************************************************************************************
TASK [Register loop output as a variable] **************************************************************************************************************************************************************************
changed: [localhost] => (item=one)
changed: [localhost] => (item=two)
changed: [localhost] => (item=three)
TASK [Print output] ************************************************************************************************************************************************************************************************
ok: [localhost] => {
“echo.results”: [
{
“ansible_loop_var”: “item”,
“changed”: true,
“cmd”: “date +%s%N ;echo one”,
“delta”: “0:00:00.007693”,
“end”: “2025-01-24 09:36:44.703260”,
“failed”: false,
“invocation”: {
“module_args”: {
“_raw_params”: “date +%s%N ;echo one”,
“_uses_shell”: true,
“argv”: null,
“chdir”: null,
“creates”: null,
“executable”: null,
“removes”: null,
“stdin”: null,
“stdin_add_newline”: true,
“strip_empty_ends”: true
}
},
“item”: “one”,
“msg”: “”,
“rc”: 0,
“start”: “2025-01-24 09:36:44.695567”,
“stderr”: “”,
“stderr_lines”: ,
“stdout”: “1737740204N\none”,
“stdout_lines”: [
“1737740204N”,
“one”
]
},
{
“ansible_loop_var”: “item”,
“changed”: true,
“cmd”: “date +%s%N ;echo two”,
“delta”: “0:00:00.007824”,
“end”: “2025-01-24 09:36:44.911104”,
“failed”: false,
“invocation”: {
“module_args”: {
“_raw_params”: “date +%s%N ;echo two”,
“_uses_shell”: true,
“argv”: null,
“chdir”: null,
“creates”: null,
“executable”: null,
“removes”: null,
“stdin”: null,
“stdin_add_newline”: true,
“strip_empty_ends”: true
}
},
“item”: “two”,
“msg”: “”,
“rc”: 0,
“start”: “2025-01-24 09:36:44.903280”,
“stderr”: “”,
“stderr_lines”: ,
“stdout”: “1737740204N\ntwo”,
“stdout_lines”: [
“1737740204N”,
“two”
]
},
{
“ansible_loop_var”: “item”,
“changed”: true,
“cmd”: “date +%s%N ;echo three”,
“delta”: “0:00:00.007903”,
“end”: “2025-01-24 09:36:45.115974”,
“failed”: false,
“invocation”: {
“module_args”: {
“_raw_params”: “date +%s%N ;echo three”,
“_uses_shell”: true,
“argv”: null,
“chdir”: null,
“creates”: null,
“executable”: null,
“removes”: null,
“stdin”: null,
“stdin_add_newline”: true,
“strip_empty_ends”: true
}
},
“item”: “three”,
“msg”: “”,
“rc”: 0,
“start”: “2025-01-24 09:36:45.108071”,
“stderr”: “”,
“stderr_lines”: ,
“stdout”: “1737740205N\nthree”,
“stdout_lines”: [
“1737740205N”,
“three”
]
}
]
}
PLAY RECAP *********************************************************************************************************************************************************************************************************
localhost : ok=2 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
You are not following what Brian tried to say. The only parallelism that exists in Ansible is to run tasks on multiple hosts in parallel. So if you want to run tasks in parallel on a single host, you have to cheat Ansible by pointing multiple hosts in inventory to a single IP address (your desired host). Have a close look at how inventory (hosts.ini) is constructed and which host selector (parallel) is used in Brian’s play.
Ok… let me try it out… Thank you for discussing on this scenario.