Is there a way to set the number of forks from within a playbook?
The default is fine for most of our playbooks, but one of them benefits from a higher number. I don't want to rely on people remembering to set -f20 for this particular playbook (that's why we use ansible, humans are unreliable!).
Anyway to do this?
Now I have another case where we install a piece of software that registers against a manager, but, that manager server can only accept one registration at a time... Being able to force "-f 1" (forks = 1 ???) would solve this issue (and the one above!).
Because we don't want to rely on human memory (remembering to use -f1 when running the playbook, or running the playbook once against each host with -l), we ended up creating one group per host - clunky!
Yves,
I think you can use the ‘serial:’ option here, maybe there’s a better way.
serial: 1 means only one host at a time
serial: 30 , 30 forks
Thanks,
Benno
Serial does not set the number of forks, it addresses the number of
hosts to manage at one time versus doing the first task across the
top.
I agree it should be made a playbook setting, but it's also
configurable in the configuration file, so this should be sufficient
for most.
--Michael
Ah! I've just done a bunch of tests....
-it works well to limit the number of forks on tasks (serial < 5)
-it never works on gather_facts. gather_facts is always run in parallel, five hosts at a time regardless of the serial option (unless -f is used)
-it does not work if you set serial > 5, it still runs 5 hosts at once (both for gather_facts and a task)
serial is the construct used to define rolling updates behavior.
and only that.
But aren't the settings in the configuration file applied to all the playbooks?
I'm looking for a set'table fork on a per playbook basis (I only have a few playbooks where I need a high fork number).
I understand.
You may wish to try Brian Coca's trick of putting a shebang line in
your playbook.
I thought that was pretty clever!
#!/usr/bin/ansible-playbook --forks 20
etc
I agree it should be a feature, please open a github ticket so we can track it.
--Michael
Is -f an upper limit or an absolute number of processes started?
If I have a group of host with 3 hosts in it, will ansible-playbook spin up 5 processes anyway (assuming forks=5), or will it only fork what he needs?
I'm trying to figure out the disadvantages of setting a high number of fork for all playbooks, even the ones that don't necessarily need it.
Thanks.
forks just uses multiprocessing.py straight, so it will create 5 if --forks=5.
Given sufficient memory, --forks 50 is not unheard of, but can get a
little RAM hungry at those levels.
Generally folks are almost always setting the serial value at those
levels (rolling update mode) and it does not make sense to have forks
exceed the serial count.
There really isn't a reason we can't auto-minimize the fork count when
it is not needed.
--Michael
Are there any new considerations on this?
Being able to specify a number of forks in a play would be really helpful.
My use case is that in a playbook I have a play to ‘git pull’ and some other plays (compile the code, for example).
The thing is that the git server I use has some restrictions on the number of simultaneous connections.
So I’d like to run the first play with a certain number of forks and the rest as much parallel as possible.
The workaround for me would be to use ‘serial’, but it runs slowly as it waits all the hosts in a series and only then runs the next one.
Nothing at this time.