# Sharing code between modules **URL:** \ **Category:** Ansible Developer\ **Created:** [October 5, 2015, 12:24am UTC](https://forum.ansible.com/t/sharing-code-between-modules/19436 "2015-10-05T00:24:16Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Marc\_Abramowitz](https://avatars.discourse-cdn.com/v4/letter/m/a5b964/32.png) [@Marc\_Abramowitz](https://forum.ansible.com/u/Marc_Abramowitz)\ **Post date:** [October 5, 2015, 12:24am UTC](https://forum.ansible.com/t/sharing-code-between-modules/19436/1 "2015-10-05T00:24:16Z")
I'm writing a set of modules that do a bunch of things with the TeamCity CI server. I have 2 such modules so far and they use some common code - currently these two modules have a lot of copied code and it's already getting a little troublesome to modify the same code in 2 places. I'd prefer to have these modules import from some common module, but ansible modules are kind of a special beast because they get transferred to the remote server. There is some special magic that runs server-side which detects imports of special things like ansible.module\_utils and actually copies the contents of the utils stuff into the module code before copying it to the remote machine. From a glance this looks to be specific to ansible.module\_utils and other internal stuff like that. I'm wondering if there has been any talk of extending this so that it could be more general and could be used to import things that don't live in ansible core? For example, I'd have in a "library" directory: teamcity\_common.py teamcity\_project.py teamcity\_build\_config.py The latter 2 would be able to import from the first one. I'm not sure I would choose to intercept "import" as changing the behavior of a built-in Python statement strikes me as confusing. I'd probably create some kind of pragma that gets embedded in a Python comment - e.g.:     ##ansible: include ansible.module\_utils, teamcity\_common Any suggestions would be appreciated. Marc ---
**Author:** ![Marc\_Abramowitz](https://avatars.discourse-cdn.com/v4/letter/m/a5b964/32.png) [@Marc\_Abramowitz](https://forum.ansible.com/u/Marc_Abramowitz)\ **Post date:** [October 5, 2015, 2:15am UTC](https://forum.ansible.com/t/sharing-code-between-modules/19436/2 "2015-10-05T02:15:51Z")
gic that I referred to before is probably well-known to the Ansible core developers and people who have poked around in internals a fair amount, but for those who aren't familiar with it but are curious: [https://github.com/ansible/ansible/blob/devel/lib/ansible/executor/module\_common.py](https://github.com/ansible/ansible/blob/devel/lib/ansible/executor/module_common.py) My proposal is essentially to allow module developers a way to get some of this magic, by using an explicit directive, such as: ##ansible: include teamcity\_common which would copy the contents of teamcity\_common.py into the current module at that point. I chose this weird syntax over `import` in an attempt to make it clearer that what this does is something special and ansible-specific and it's not an actual Python `import`, as it's doing text substitution and doesn't really involve the `PYTHONPATH` and such. ---
**Author:** ![sivel](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/sivel/32/1198_2.png) [@sivel](https://forum.ansible.com/u/sivel)\ **Post date:** [October 5, 2015, 2:17am UTC](https://forum.ansible.com/t/sharing-code-between-modules/19436/3 "2015-10-05T02:17:16Z")
Ansible can already do this, but it currently requires that the shared code be copied into the module\_utils directory in your Ansible installation. Your other option is to make it a real Python module, and install it on the remote nodes so that your Ansible modules can import it. ---
**Author:** ![Marc\_Abramowitz](https://avatars.discourse-cdn.com/v4/letter/m/a5b964/32.png) [@Marc\_Abramowitz](https://forum.ansible.com/u/Marc_Abramowitz)\ **Post date:** [October 5, 2015, 2:32am UTC](https://forum.ansible.com/t/sharing-code-between-modules/19436/4 "2015-10-05T02:32:30Z")
Yeah those are the options I considered for now and I had one other which is that I could write my code in separate files and then have a Makefile or something that concatenates them together, essentially doing what Ansible does, but doing it a "compile-time". I guess if my modules turned out to be generally useful and good, then I could submit a PR and that could include a file that gets installed in the module\_utils directory. But long-term it seems a little limiting that something has to go into core to be able to do that, so I wonder if there would be interest in a PR that would extend the magic a bit as previously mentioned? ---
**Author:** ![John\_Burwell](https://avatars.discourse-cdn.com/v4/letter/j/b77776/32.png) [@John\_Burwell](https://forum.ansible.com/u/John_Burwell)\ **Post date:** [October 5, 2015, 5:02pm UTC](https://forum.ansible.com/t/sharing-code-between-modules/19436/5 "2015-10-05T17:02:01Z")
All, Long time listener, first time caller. What about a decorator on the module class to indicate the extra modules/assets required to be transferred with a module? Thanks, -John ---
**Author:** ![Marc\_Abramowitz](https://avatars.discourse-cdn.com/v4/letter/m/a5b964/32.png) [@Marc\_Abramowitz](https://forum.ansible.com/u/Marc_Abramowitz)\ **Post date:** [October 12, 2015, 12:18am UTC](https://forum.ansible.com/t/sharing-code-between-modules/19436/6 "2015-10-12T00:18:57Z")
That's an interesting idea. I'm not a core dev though so I'd wait for one of them to weigh in before playing with it myself. ---
**Author:** ![Toshio\_Kuratomi](https://avatars.discourse-cdn.com/v4/letter/t/41988e/32.png) [@Toshio\_Kuratomi](https://forum.ansible.com/u/Toshio_Kuratomi)\ **Post date:** [October 13, 2015, 11:09pm UTC](https://forum.ansible.com/t/sharing-code-between-modules/19436/7 "2015-10-13T23:09:09Z")
I don't think the decorator would work except as something else that overrides real python syntax to specify substitutions when the module is constructed to send over the wire. So it wouldn't actually gain us anything. If you think that it could work differently feel free to expand on the idea 🙂 We have talked about a separate directory to search for module\_utils type snippets of code using the same mechanism as ansible's current replacer. Not a high priority because: 1) Most sites can already do this simply by dropping their extra files into /usr/lib/python2.7/site-packages/ansible/module\_utils/ (use the path that is appropriate for your ansible install) Since the only thing that makes the asible control machine special is that you have access to credentials on that machine this covers most use cases (there are a few that it doesn't -- where you don't have privileges to modify that directory and are not allowed by policy from running from a checkout/separate tarball/your own machine). 2) I personally want to deprecate replacer in favour of real imports (from a limited directory structure).I've got some code that starts down this path but it definitely won't make the cut for 2.0... Possibly something for 2.1. This isn't mutually exclusive with people having a directory that parallels module\_utils but I wouldn't want people to start adopting bad habits due to relying on what replacer does rather than what a real import would do. As for what tokens to use to activate module replacer... probably the most natural is a second import that looks like the current one but uses a different directory. For instance: "from ansible.user\_module\_utils.FOO import \*" I share your dislike of overloading import but it's not worse than what we already have. If you want to go the comment route, replacer used to use comments before the "from ansible.module\_utils[..] import \*" string was created. It used (and still does although it's deprecated): "#\<\\>" to substitute basic.py into a module. Maybe that could be expanded to more than basic.py or perhaps it is best consigned to the dustbin as it's better to have just one thing that people have to know than two that do the same thing (and at this point, we can't get rid of the present replacer syntax for backwards compatibility reasons). If you're interested in ziploader, I put together some pieces to prove it could work last time I had some vacation: [https://gist.github.com/abadger/d3592c1c9ef37ca54db0](https://gist.github.com/abadger/d3592c1c9ef37ca54db0) That's a standalone script that creates a file which can be executed by a python interpreter via a pipeline or as an argument on the command line. it's one of maybe three parts that would have to come together to make a working proof of concept. The others are changing modules\_common modify\_module code so that we detect when a module needs to use a snippet from module\_utils and then add it to the zip file's payload (and recursively scan those files for other module snippets from module\_utils) and then plugging all this into the existing framework so that certain things can be configured from hostvars (compressing the zip data is likely something that needs to be configurable in case a remote host doesn't have a python with zlib), making it coexist with the existing replacer strategy for backwards compatibility, and then modifying the module\_utils and module code so that it can be used by zip\_loader instead of replacer (mostly changing things so we don't rely on variables being present at the global scope). The latter is the kind of thing I mean when I talk about not wanting a user-expandable directory to allow bad habits. There's a few ways to make ziploader coexist with replacer. A few weeks ago I talked with a few other people about making replacer only respond to "from ansible.module\_utils.FOO import \*" (which is what it uses now) and letting all other things that import from ansible.module\_utils use the real import mechanism (from ansible.module\_utils import FOO and from ansible.module\_utils.FOO import BAR). Another way would be to make things explicit: "from ansible.module\_utils.FOO import \*" would use replacer "from ansible.module\_utils2.FOO import \*" would use real imports. I think #2 will lead to duplication of module\_utils code so I currently am favoring #1. Right now ziploader is on hold for me -- lots of work to do to get ansible-2.0 out the door so I can't be spending too much time on 2.1 stuff yet. -Toshio