# Seeking testers for a (radically) performance-improving Ansible plug-in

**URL:** <https://forum.ansible.com/t/seeking-testers-for-a-radically-performance-improving-ansible-plug-in/26763>\
**Category:** Ansible Project\
**Created:** [February 16, 2018, 2:02pm UTC](https://forum.ansible.com/t/seeking-testers-for-a-radically-performance-improving-ansible-plug-in/26763 "2018-02-16T14:02:49Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![dw1](https://avatars.discourse-cdn.com/v4/letter/d/c6cbf5/32.png) [@dw1](https://forum.ansible.com/u/dw1)\
**Post date:** [February 16, 2018, 2:02pm UTC](https://forum.ansible.com/t/seeking-testers-for-a-radically-performance-improving-ansible-plug-in/26763/1 "2018-02-16T14:02:49Z")

</div>

Hi guys,

I’ve been working occasionally on a new connection method for Ansible that drastically improves its efficiency, even when compared to pipelining and SSH multiplexing. The typical speedups can be impressive - in the 1.5x to 5x range, and some nice side effects exist such as system logs not being spammed while Ansible executes.

[http://mitogen.readthedocs.io/en/latest/ansible.html](http://mitogen.readthedocs.io/en/latest/ansible.html)

It takes all of 5 minutes over lunch to try out, and I’d love to get feedback on it from a variety of playbooks, if in no other form than bug reports. 😉

Thanks,

David

---

<div class="post-metadata">

**Author:** ![dawgfoto](https://avatars.discourse-cdn.com/v4/letter/d/b9e5f3/32.png) [@dawgfoto](https://forum.ansible.com/u/dawgfoto)\
**Post date:** [February 17, 2018, 2:25pm UTC](https://forum.ansible.com/t/seeking-testers-for-a-radically-performance-improving-ansible-plug-in/26763/2 "2018-02-17T14:25:55Z")

</div>

Very interesting, just brought up [https://github.com/ansible/ansible/issues/36275](https://github.com/ansible/ansible/issues/36275).

Mitogen (or basically any short-lived agent) seems like a feasible approach to solve the problem, albeit sending python modules via ssh and executing them in a long(er)-running process adds quite some complexity.  
What’s wrong with just caching python modules (or binaries) in a size-limited temp folder? Using content-addressing should get rid of possible versions conflicts and a tiny bit of disk or tmpfs space doesn’t seems like an unreasonable requirement.

---

<div class="post-metadata">

**Author:** ![dw1](https://avatars.discourse-cdn.com/v4/letter/d/c6cbf5/32.png) [@dw1](https://forum.ansible.com/u/dw1)\
**Post date:** [February 18, 2018, 2:52am UTC](https://forum.ansible.com/t/seeking-testers-for-a-radically-performance-improving-ansible-plug-in/26763/3 "2018-02-18T02:52:35Z")

</div>

It’s certainly possible to cache modules, the problem is that aside from requiring a writeable disk, it introduces intermediate persistent state, which is evil in any software design where it can otherwise be avoided, as it creates all kinds of coherency and management issues (of which you name one - limiting the size!), and greatly increases the number of generally difficult/impossible to test states the software may find itself in.

I’m not averse to adding it, but meanwhile there are more ripe performance hindrances to address beforehand.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/ansible/original/1X/1882d9a01da5d8996a862359b0913cc568a39b84.svg) [@system](https://forum.ansible.com/u/system)\
**Post date:** [February 19, 2018, 6:36pm UTC](https://forum.ansible.com/t/seeking-testers-for-a-radically-performance-improving-ansible-plug-in/26763/4 "2018-02-19T18:36:34Z")

</div>

So during vacation i was toying with an 'ansible remote execution'  
(rme) [https://github.com/bcoca/ansible/tree/rme](https://github.com/bcoca/ansible/tree/rme) , via a connection  
plugin that requires Ansible be installed on the target node.

It could either have a service/socket/trigger to run ansible tasks as  
passed in or we can just execute `ansible -m module ...` directly.

This avoids 'cache invalidation issues' but does introduce version  
mismatch ones as well as 'agent or required software' on the targets.  
More thorny issues are 'what do we pass', templated task? task loop?  
original task + vars? what secrets do we pass and how?

I have not seen an approach that does not have drawbacks, in the end  
we might implement more than one and let the user choose their  
tradeoffs.