# Protocol is slow **URL:** \ **Category:** Ansible Project\ **Created:** [September 26, 2012, 4:29pm UTC](https://forum.ansible.com/t/protocol-is-slow/11929 "2012-09-26T16:29:44Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Aleksej\_Romanov](https://avatars.discourse-cdn.com/v4/letter/a/278dde/32.png) [@Aleksej\_Romanov](https://forum.ansible.com/u/Aleksej_Romanov)\ **Post date:** [September 26, 2012, 4:29pm UTC](https://forum.ansible.com/t/protocol-is-slow/11929/1 "2012-09-26T16:29:44Z")
I like Ansible's approach to management. Don't need almost anything at managed hosts. Don't need any central daemon. Copy the module, run, get results, remove, repeat. Easy. But unfortunately can be slow. I have a playbook with ~200 actions. It takes 20 seconds to run with '-c local'. About a minute if use SSH to localhost. Connecting and disconnecting for every step really makes things slower... And 12 minutes if used with remote server. Shared SSH connections (ControlMaster / ControlPersist) are enabled, servers aren't too busy but located in different datacenters. ansible-pull is nice, but not really what I would like. I don't need to run tasks via Cron, I need to trigger them manually. Also will need to have central Git repository for playbooks. Being able to run Ansible, look at output and fix problems if any is much more desirable than getting reports about errors by e-mail or something similar. Don't need to tell. Everyone here knows advantages of usual Ansible usage way 🙂 So. Aren't there any plans to change usual process which now requires additional connections and steps? I understand that it helps to make thing more easy, but anyway... Require Ansible to be installed on remote computer, copy playbook, run with '-c local' and show results in the same way as now? Or just copy Ansible there together with playbook and remove when execution finished? Don't copy full Ansible, but some smaller (and having less dependencies) script and provide it with playbook already parsed at central server (I dont't know if this is reasonable)? Quite big changes. A bit simpler one would be to create temp directory and upload all modules only once, don't repeat this for every 'action' line in playbook. As for now one can see in debug (-vvv) output 5 actions related to network activity: \<127.0.0.1\> EXEC mkdir -p $HOME/.ansible/tmp/ansible-1348675481.15-70644186131164 && chmod a+rx $HOME/.ansible/tmp/ansible-1348675481.15-70644186131164 && echo $HOME/.ansible/tmp/ansible-1348675481.15-70644186131164 \<127.0.0.1\> REMOTE\_MODULE setup \<127.0.0.1\> PUT /tmp/tmpLYtBRt TO /home/alopropoz/.ansible/tmp/ansible-1348675481.15-70644186131164/setup \<127.0.0.1\> EXEC chmod u+x /home/alopropoz/.ansible/tmp/ansible-1348675481.15-70644186131164/setup \<127.0.0.1\> EXEC /home/alopropoz/.ansible/tmp/ansible-1348675481.15-70644186131164/setup \<127.0.0.1\> EXEC rm -rf /home/alopropoz/.ansible/tmp/ansible-1348675481.15-70644186131164/ As for me, creating directory, uploading module, making it executable and removing directory can be done only once for playbook, so eliminating 4 of 5 network-related actions. Does this make sense? ---
**Author:** ![Chamberlain\_Darren](https://avatars.discourse-cdn.com/v4/letter/c/ee7513/32.png) [@Chamberlain\_Darren](https://forum.ansible.com/u/Chamberlain_Darren)\ **Post date:** [September 26, 2012, 4:42pm UTC](https://forum.ansible.com/t/protocol-is-slow/11929/2 "2012-09-26T16:42:11Z")
You can use a hybrid approach: Install ansible-pull on each system, but instead of triggering it via cron, trigger it using ansible on your control node: ---
**Author:** ![Michael\_DeHaan](https://avatars.discourse-cdn.com/v4/letter/m/c4cdca/32.png) [@Michael\_DeHaan](https://forum.ansible.com/u/Michael_DeHaan)\ **Post date:** [September 26, 2012, 5:20pm UTC](https://forum.ansible.com/t/protocol-is-slow/11929/3 "2012-09-26T17:20:16Z")
Replies inline -- lots of good ideas, some we can't really do, but many we can. Read on for details. I've been waiting for things to stabilize with the new module logic and I think we are at a point where we can start doing a lot of this kind of stuff to core again. > I like Ansible's approach to management. Don't need almost anything at > managed hosts. Don't need any central daemon. Copy the module, run, get > results, remove, repeat. Easy. > > But unfortunately can be slow. > > I have a playbook with ~200 actions. It takes 20 seconds to run with > '-c local'. About a minute if use SSH to localhost. Connecting and > disconnecting for every step really makes things slower... And 12 minutes > if used with remote server. Shared SSH connections (ControlMaster / > ControlPersist) are enabled, servers aren't too busy but located in > different datacenters. BTW -- If you aren't already using with\_items with your package installs, definitely switch to that. If your 12 minutes includes a lot of package installations I also would think that's reasonably fast and you may be seeing some speedups locally because you're using a local yum mirror? (Perhaps not?) > ansible-pull is nice, but not really what I would like. I don't need to > run tasks via Cron, I need to trigger them manually. Also will need to > have central Git repository for playbooks. Being able to run Ansible, > look at output and fix problems if any is much more desirable than > getting reports about errors by e-mail or something similar. > > Don't need to tell. Everyone here knows advantages of usual Ansible > usage way 🙂 > > So. Aren't there any plans to change usual process which now requires > additional connections and steps? I understand that it helps to make > thing more easy, but anyway... A mode that uses paramiko and leaves connections open in configurable (Least-Recently-Used way is being considered right now. Given paramiko and SSH with ControlMaster/ControlPersist are on par right now, this has a lot of promise if you are willing to set the number of open connections reasonably high (i.e. equal to your number of hosts in a play). > Require Ansible to be installed on remote computer, copy playbook, run > with '-c local' and show results in the same way as now? Or just copy > Ansible there together with playbook and remove when execution finished? This is basically what ansible-pull does, and you wouldn't have to run it out of cron. You could use ansible to invoke ansible-pull, in fact. > Don't copy full Ansible, but some smaller (and having less dependencies) > script and provide it with playbook already parsed at central server (I > dont't know if this is reasonable)? > > Quite big changes. A bit simpler one would be to create temp directory > and upload all modules only once, don't repeat this for every 'action' > line in playbook. As for now one can see in debug (-vvv) output 5 > actions related to network activity: Yeah this is a non-starter because of modules needing to think about results to decide what to run next. > \<127.0.0.1\> EXEC mkdir -p > $HOME/.ansible/tmp/ansible-1348675481.15-70644186131164 && chmod a+rx > $HOME/.ansible/tmp/ansible-1348675481.15-70644186131164 && echo > $HOME/.ansible/tmp/ansible-1348675481.15-70644186131164 > \<127.0.0.1\> REMOTE\_MODULE setup > \<127.0.0.1\> PUT /tmp/tmpLYtBRt TO > /home/alopropoz/.ansible/tmp/ansible-1348675481.15-70644186131164/setup > \<127.0.0.1\> EXEC chmod u+x > /home/alopropoz/.ansible/tmp/ansible-1348675481.15-70644186131164/setup > \<127.0.0.1\> EXEC > /home/alopropoz/.ansible/tmp/ansible-1348675481.15-70644186131164/setup > \<127.0.0.1\> EXEC rm -rf > /home/alopropoz/.ansible/tmp/ansible-1348675481.15-70644186131164/ > > As for me, creating directory, uploading module, making it executable > and removing directory can be done only once for playbook, so eliminating > 4 of 5 network-related actions. Does this make sense? Actually this is QUITE true. It's not done because failed hosts are not contacted again, but if cleanup is added in a try/except/finally sort of way and the decision of what temp dir name to use is made in PLAYBOOK code and passed to Runner this could be beautiful, and is QUITE easy to do. The other hack is that if we know what the interpreter to use is (AND WE DO, we look at ansible\_python\_interpreter and the shebang line), there is no need for the chmod +x as we can just directly execute the module. Another trick I have not added yet is that if we know we are executing only new style modules (those written using the ANSIBLE\_MODULE\_COMMON code) and they do not involve file transfer, there is no need for a directory, we can just move a \*file\* and execute it. That would carve out about half of the operations right there. ---
**Author:** ![Michael\_DeHaan](https://avatars.discourse-cdn.com/v4/letter/m/c4cdca/32.png) [@Michael\_DeHaan](https://forum.ansible.com/u/Michael_DeHaan)\ **Post date:** [September 26, 2012, 5:22pm UTC](https://forum.ansible.com/t/protocol-is-slow/11929/4 "2012-09-26T17:22:18Z")
>   - hosts: all >     user: root >     tasks: >       - name: do stuff >         action: shell ansible-pull ... > > That way you still get the on-demand control but each system runs > independantly. You still have the (potential) problem of collecting > the logs, but that's an easier problem to solve. Yep. The fetch module is pretty good for this! Analyzing them is a bit more of an effort, but also not insurmountable. There is a patch I need to clean up that will also allow per hostname top level playbooks for ansible-pull instead of just requiring them to be named local, or you can also pass in the name of the playbook file in the repo. ---
**Author:** ![Aleksej\_Romanov](https://avatars.discourse-cdn.com/v4/letter/a/278dde/32.png) [@Aleksej\_Romanov](https://forum.ansible.com/u/Aleksej_Romanov)\ **Post date:** [October 3, 2012, 8:53pm UTC](https://forum.ansible.com/t/protocol-is-slow/11929/5 "2012-10-03T20:53:23Z")
I think I will stick with this as for now. Not ansible-pull but just ansible, though, to avoid the need to maintain central repository with playbooks available for all hosts. Central server can copy playbook, run ansible and wait for results. Since (hopefully) most of time there will be no errors and Ansible will let me know in case if an error happen, I will start without collecting logs. Thank you for the nice idea. ---
**Author:** ![Aleksej\_Romanov](https://avatars.discourse-cdn.com/v4/letter/a/278dde/32.png) [@Aleksej\_Romanov](https://forum.ansible.com/u/Aleksej_Romanov)\ **Post date:** [October 3, 2012, 9:04pm UTC](https://forum.ansible.com/t/protocol-is-slow/11929/6 "2012-10-03T21:04:37Z")
> BTW -- If you aren't already using with\_items with your package > installs, definitely switch to that. > > If your 12 minutes includes a lot of package installations I also > would think that's reasonably fast and you may be seeing some speedups > locally because you're using a local yum mirror? (Perhaps not?) I use 'with\_items', thank you. But this is not first run of playbook. Everything is configured, all packages are installed. No actual actions are executed (except may be few simple shell commands). Difference between 20 seconds for '-c local' and 12 minutes for remote machine is all because of network interactions. > \> Require Ansible to be installed on remote computer, copy playbook, run > \> with '-c local' and show results in the same way as now? Or just copy > \> Ansible there together with playbook and remove when execution finished? > > This is basically what ansible-pull does, and you wouldn't have to run > it out of cron. You could use ansible to invoke ansible-pull, in > fact. Now it would be executed like any other external program: you wait until ansible on remote side is done and get a lot of output. From first host, from second and so on. Quite difficult to understand. Usual ansible mode (with results shown after each task) is much more handy. Don't know if there is a point in implementing this though. ---
**Author:** ![Michael\_DeHaan](https://avatars.discourse-cdn.com/v4/letter/m/c4cdca/32.png) [@Michael\_DeHaan](https://forum.ansible.com/u/Michael_DeHaan)\ **Post date:** [October 3, 2012, 9:06pm UTC](https://forum.ansible.com/t/protocol-is-slow/11929/7 "2012-10-03T21:06:38Z")
I think this thread is sufficiently older than fireball mode that I should mention it: You should read these: [http://michaeldehaan.net/post/32378722265/ansible-learns-to-fly-0mq-that-sets-up-itself](http://michaeldehaan.net/post/32378722265/ansible-learns-to-fly-0mq-that-sets-up-itself) [http://jpmens.net/2012/10/01/dramatically-speeding-up-ansible-runs/](http://jpmens.net/2012/10/01/dramatically-speeding-up-ansible-runs/) ---
**Author:** ![Michael\_DeHaan](https://avatars.discourse-cdn.com/v4/letter/m/c4cdca/32.png) [@Michael\_DeHaan](https://forum.ansible.com/u/Michael_DeHaan)\ **Post date:** [October 3, 2012, 9:08pm UTC](https://forum.ansible.com/t/protocol-is-slow/11929/8 "2012-10-03T21:08:28Z")
> But this is not first run of playbook. Everything is configured, all > packages are installed. No actual actions are executed (except may be > few simple shell commands). Difference between 20 seconds for '-c local' > and 12 minutes for remote machine is all because of network > interactions. Sounds like you need to be running ansible on a head-node inside your target network instead of from really far away and over some slow pipes?