long term optimization idea

Archives Ansible Developer
1

Hello,

regarding the number of ssh roundrips, ssh_alt lowers them down for all modules that do not need files transfered (eg template, copy)

the overall principle of the execution path of a module is

  • module = render(module, data) # all ‘new’ modules work this way
  • cat module | ssh -c python

when there are several files to send you cannot do this.

but as you may know, python since 2.6 knows how to execute zip archives (egg are zip archives)

so you could imagine that a zip file is prepared will all python files necessary, all ressources and this file is send over the network.

on the principle a sort of

  • module.egg = render(module, data)
  • cat module.egg | ssh -c python

the problem is that this does not work because “python module.egg” works but not “cat module.egg | python”. I did some research on this that you can read here http://stackoverflow.com/questions/20276105/why-cant-python-execute-a-zip-archive-passed-via-stdin
(you can give some points to the guy who answered this thread he deserves it :wink:

with his solution, considering that one “bootstrap.py” file is sent to a remote at the beginning of a play, then you can do

  • module.egg = render(module, data)
  • cat module.egg | ssh -c python bootstrap.py

=> many more operations can be done with 1 ssh roundrip
=> full python archives can be sent simplifying the rendering of the modules (no need to template the data inside the module for example).
=> modules could be more easily decomposed into building blocks

Just sending this in case it gives you some ideas & in order that the stack overflow answer does not get lost.

As the guy said :
" but I cannot imagine any engineer thinking that it would be a good solution for how to handle code updates or remote control."

so i guess I am totally assuming the role of the engineer here :wink:

2

So this zipfile/tarball idea is very very similar to one that Seth Vidal and I were throwing around in the very early days of the project, calling this idea (jokingly) the “ansi-ball”.

It’s not entirely a fit immediately right now in the present implementation – because the modules contain their arguments now – but I’m not opposed to experimentation and by no means must things continue to work exactly as they do now.

Rather than having files with just arguments, or would at least require a wholesale change to achieve this. Rather than a module having backed-in arguments, or transferring a modules file, it would have to feed in module arguments to the modules via stdin.

It would also require removing support for dynamically decided module names, a la “action: $ansible_pkg_mgr”, but I’m ok with that if it proves shiny.

" " but I cannot imagine any engineer thinking that it would be a good solution for how to handle code updates or remote control."

I suspect it’s about the same level of efficiency, but I also don’t think a blanket statement should be made it’s going to be a bad idea.

We would of course have to preserve the existing code-path for non-Python-native modules.

Should this happen though, I’d rather see this happen via Unix toolchain than python egg facilities, just as we might be more likely to encounter something version specific.

One complexity might come with sudo support and different users, needing to transfer things to different locations, and possibly different balls, so I suspect there’s a chance this might not be as theoretically useful in the end.