Can I provide .pyo files as module_path library

Archives Ansible Project
1

Hi Guys.

currently we are shipping our code as .pyo to ensure a minimal security to our python code base. I wanted to know if passing the module_path=‘library_path’ where the library path has

.pyo files instead of .py files work, since I was getting an error saying module not found.

Can someone please comment on this?

Thanks
Poojithaa

2

Poojithaa,

You’re probably encountering the error here: https://github.com/ansible/ansible/blob/devel/lib/ansible/executor/module_common.py#L512

Due to possible version differences between the python version used to compile the module and the python version on the remote system which will execute the module, modules must be available as source.

  • Matt
3

Hi Matt,

Could be the case however, we are running all the files locally. It should be using the same compiler that we used to build .pyo files.

4

Ansible is not designed to handle this case. There’s too many things that become impossible. I can think of a work around, though.

The workaround is to make your modules into a binary modue. Something like cx_freeze will create a single file executable with all of the things necessary to execute the code ( I’d recommend looking at this page, picking a tool that might do what you need and then reading that tool’s documetation for more information: http://docs.python-guide.org/en/latest/shipping/freezing/ ) The single file executable can then be used by ansible as a binary module (one that can be executed directly from the shell on the remote machine) rather than having to try pushing it through an interpreter.

As for some of the reasons Ansible cannot support directly executing a byte compiled file:

  • We can’t scan .pyos or .pycs to determine what module_utils files need to be included when we send the module over the wire.
  • byte compiled files aren’t directly executable so they need to be run by an interpreter. But the byte compiled file doesn’t have a shebang line so we can’t detect what interpreter to use.
  • byte compiled files are restricted to the python major version so we can’t know whether they’ll actually run on the remote machine’s python
  • .pycs and .pyos don’t actually provide any security… at best they’re a mild, reversible obfuscation. Someone who’s taken the trouble to acquire the byte compiled files can easily reverse them to find out what’s going on.

-Toshio

5

Since python bytecode can be easily converted back to python source, there isn’t any security there.

That aside, and I don’t recommend this, you could create a python stub loader which loaded embedded python bytecode. Here’s a quick proof of concept (again, I don’t recommend this):

#!/usr/bin/python

import base64
import marshal

with open(‘module.pyc’) as fd:

bytecode = base64.b64encode(fd.read()[8:])

bytecode = ‘INSERT YOUR BYTECODE HERE, SEE COMMENT ABOVE’

exec marshal.loads(base64.b64decode(bytecode))

  • Matt