Not forking parrallel tasks?

Archives Ansible Project
1

I have found when running ansible with either playbooks or direct commands that the forking is not working as I’d expect. If I add -f 20 or -f 3 or --forks #, it appears I still end up with only 1 remote command running at a time. I check this by looking at the processes running on the ansible server and I only see 1 remote ssh session at a time to my remote machines.
I am running an older version of ansible 1.2.2 because it is the latest in the Ubuntu LTS repo. Has anyone else seen this? Is there a problem with my syntax?

Thanks

2

Forks are used for multiple servers, not multiple commands per server.

So if you are operating over 30 servers, you could set 30 forks and run a single command across all servers at once.

3

Correct. I am running a single command (such as apt-get update) against a dozen or so servers and want it to happen in parallel. Unfortunately, when using --forks 20 or -f 20 I still only get connections to 1 server at a time. They run consecutively and not concurrently.

4

sounds like you have ‘serial: 1’ set in the playbook.

5

This happens whether I use ansible-playbook or ansible. I do not have serial:1 (or any serial setting) in my playbooks or my ansible.cfg

6

For example, something as simple as:

ansible MyServers -m shell -a "uptime " -f 20

Only returns the uptime for server at a time. I have also tried using a sleep statement for example and only a single server returns its results at a time after the sleep interval. They are not running in parallel for some reason.

7

output to your screen is serialized (otherwise it would be intermixed) but
the actual remote actions should be in parallel.

8

The don’t appear to be however, that is my concern. For example I assume a command like:

ansible MyServers -m shell -a “sleep 20” -f 20

When run on a list of less than 20 servers should return output from all servers around 20 seconds after it is initiated. It is not. 1 server replies, 20 seconds later the next server replies, 20 seconds later the next server replies etc.

They are not running in parallel. What am I missing?

9

just tested :

ansible all -m shell -a “sleep 10 && uptime” -f 10

and i get batches of 10 every 10 seconds (as expected)
tested with git checkout and version 1.3.3

10

That is the expected behavior I was also hoping to see. But I do not however. How can I trouble shoot this?

11

What connection method are you using? What verification steps are you taking to count the number of simultaneous connections?

12

start with -vvvv, also I’ll check 1.2.2 … there might be a locking issue that has been fixed in newer versions.

13

I have not manually change any connection method so I assume it should still be the default paramiko with shared SSH keys. Here is a test showing that it does not run in parallel. Notice that each response is 10 seconds after the previous one even though -f 10 is specified.

ansible Dev -m shell -a “sleep 10 && date” -f 10
pdx-cass-d02 | success | rc=0 >>
Thu Nov 7 22:54:39 UTC 2013

pdx-extws-d02 | success | rc=0 >>
Thu Nov 7 22:54:49 UTC 2013

pdx-intws-d01 | success | rc=0 >>
Thu Nov 7 22:55:00 UTC 2013

pdx-extws-d01 | success | rc=0 >>
Thu Nov 7 22:55:10 UTC 2013

pdx-fep-d01 | success | rc=0 >>
Thu Nov 7 22:55:20 UTC 2013

pdx-cass-d01 | success | rc=0 >>
Thu Nov 7 22:55:30 UTC 2013

pdx-lb-d01 | success | rc=0 >>
Thu Nov 7 22:55:40 UTC 2013

pdx-mq-d01 | success | rc=0 >>
Thu Nov 7 22:55:51 UTC 2013

pdx-sql-d01 | success | rc=0 >>
Thu Nov 7 22:57:35 UTC 2013

pdx-job-d01 | success | rc=0 >>
Thu Nov 7 22:56:11 UTC 2013

pdx-listen-d01 | success | rc=0 >>
Thu Nov 7 22:56:21 UTC 2013

pdx-sql-d02 | success | rc=0 >>
Thu Nov 7 22:58:05 UTC 2013

14

So using -vvvv it appears that ansible first connects to ALL servers to copy the tmp file for execution and the password files etc. It then connects individually to the first server to execute that command. Once a response is returned it then connects to the next server.
The verbose output is quite extensive. I’m not sure what else I should be looking for…

15

Are you able to reduce this down to a simple playbook or a module call and reproduce the behavior?

16

That is what I was hoping to demonstrate with my simple module call of:

ansible Dev -m shell -a “sleep 10 && date” -f 10

Each result comes back 10 seconds after the previous result.

pdx-cass-d02 | success | rc=0 >>
Thu Nov 7 22:54:39 UTC 2013

pdx-extws-d02 | success | rc=0 >>
Thu Nov 7 22:54:49 UTC 2013

pdx-intws-d01 | success | rc=0 >>
Thu Nov 7 22:55:00 UTC 2013

pdx-extws-d01 | success | rc=0 >>
Thu Nov 7 22:55:10 UTC 2013

17

so I tested with 1.2.2 and 1.2.3 and cannot reproduce the issue:

ansible all -m shell -a “sleep 10 && uptime” -f 10

it returns 10 hosts every 10s, as expected

18

I will upgrade to ansible 1.3.3 today and see if that doesn’t solve my issues.
Thanks

19

FYI- Upgrading to 1.3.3 did the trick. I’m not sure why I had the issue earlier that was unreproducible by others.
Thanks for your help everyone.

20

Posting on an old thread, since I just had this same issue with ansible 1.9.2.
In the end, I could solve it removing my persisted ssh connections like this
rm ~/.ansible/cp/*
I have no idea why exactly this solved the problem but in case you encounter the same issue you can try it.