python executable zip files as ansible modules

If ansible modules truly can be written in any language,
I would like that language to include “python executable zip files:slight_smile:
so that my ansible modules can refer to python modules
that I bundle with them.

I’ve prototyped changes that support this with just a few
simple changes that prevent ansible from corrupting zip files
during module snippet templating and transfer.
I think these changes could be relevant for any “language” supported by ansible,
not just zip files. Ansible should modify a module only when a user wants
it to.

A python zip bundle would still specify the python interpreter in
its shbang line, as described in the preceding link.

I would like the ability to piggyback my own application-specific python
modules along with an ansible module without having to copy-paste application-
specific code into a single ansible module source file.

Just package up an ansible module as main.py
with a few of its supporting python modules
and prepend the shbang.

I’ve implemented this with two simple code changes,
however they would impose the following new behavior on how ansible treats
modules:

  • Other than shbang interpreter replacement,
    Ansible will not change the source of a module that does not invoke the module templating mechanism.

This just seems like polite behavior for ** any ** kind of ansible module, zip or not.
Currently, ansible subtly adds a newline to the last line of a module source
file, whose last line does not end with a newline.

Adding a newline breaks the zip file format,
which locates its internal directory at the end of the file.

Would this change break some other mechanism that pipes other modules to their interpreter
on the remote system?

The code change involves ModuleReplacer._find_snippet_imports using string.splitlines
instead of string.split to split the module_data. splitlines preserves the
presence (or absence) of a termination newline.

  • Ansible will not require a module to be encoded into UTF-8 for transport to the remote system
    if its arguments are transferred in a separate file.

The arguments file would still be encoded, but does the module itself have to be?
Encoding into UTF-8 fails for zip files.

The code change involves adding an optional encode parameter (defaulting to True)
to Runner._transfer_str, so that modules may be transferred un-encoded
as long as the arguments are in a separate file.

Are there connection types that would break if the module itself
was not transferred encoded in UTF-8?

With those two changes, voila! I can use my own python modules on the remote system.
I would be happy to work out the details and contribute a pull request.

Thank you.

-tim

Adding related topics:

zip and compiled modules share similar problems with templating and/or transfer,
although zip may be more susceptible to the appended newline.

I’d be open to considering this but I’d need to see the pull request that implemented it, given I suspect very few users (no offense) would make use of this - it would depend on patch complexity.

Compiled modules in other languages I agree should be a thing.