# How to pylint and unit test module\_utils code in my local module/role? **URL:** \ **Category:** Ansible Developer\ **Created:** [March 6, 2020, 5:01pm UTC](https://forum.ansible.com/t/how-to-pylint-and-unit-test-module-utils-code-in-my-local-module-role/32408 "2020-03-06T17:01:08Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Richard\_Megginson](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/richard_megginson/32/4540_2.png) [@Richard\_Megginson](https://forum.ansible.com/u/Richard_Megginson)\ **Post date:** [March 6, 2020, 5:01pm UTC](https://forum.ansible.com/t/how-to-pylint-and-unit-test-module-utils-code-in-my-local-module-role/32408/1 "2020-03-06T17:01:08Z")
When developing a role that has a local module with some code split into module\_utils, how do you set up your pythonpath or venv so that pylint and unit testing resolves the imports correctly? For example: [https://github.com/linux-system-roles/network/](https://github.com/linux-system-roles/network/) The module code is in [https://github.com/linux-system-roles/network/blob/master/library/network\_connections.py](https://github.com/linux-system-roles/network/blob/master/library/network_connections.py) The module\_utils code is in [https://github.com/linux-system-roles/network/tree/master/module\_utils/network\_lsr](https://github.com/linux-system-roles/network/tree/master/module_utils/network_lsr) The module code imports the module\_utils code like this: # pylint: disable=import-error, no-name-in-module from ansible.module\_utils.network\_lsr import MyError As you can see, we have to disable pylint checking, because there is no ansible.module\_utils.network\_lsr - there is a module\_utils.network\_lsr however. We have to do something similar in the unit test code: [https://github.com/linux-system-roles/network/blob/master/tests/unit/test\_nm\_provider.py](https://github.com/linux-system-roles/network/blob/master/tests/unit/test_nm_provider.py) sys.modules["ansible"] = mock.Mock() sys.modules["ansible.module\_utils.basic"] = mock.Mock() sys.modules["ansible.module\_utils"] = mock.Mock() sys.modules["ansible.module\_utils.network\_lsr"] = \_\_import\_\_("network\_lsr") This seems pretty hackish, and I'm hoping there is a better or standard way to do this. ---
**Author:** ![sivel](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/sivel/32/1198_2.png) [@sivel](https://forum.ansible.com/u/sivel)\ **Post date:** [March 6, 2020, 5:19pm UTC](https://forum.ansible.com/t/how-to-pylint-and-unit-test-module-utils-code-in-my-local-module-role/32408/2 "2020-03-06T17:19:58Z")
There isn’t a great answer here. roles weren’t originally designed to work this way. You may want to look into collections instead, which have proper python imports, that aren’t effectively “fake”. A collection can include modules/plugins, as well as roles. However the location of the module moves. This doesn’t really solve things for older versions of Ansible however, as collections are a recent addition. ---
**Author:** ![AlanCoding](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/alancoding/32/301_2.png) [@AlanCoding](https://forum.ansible.com/u/AlanCoding)\ **Post date:** [March 6, 2020, 5:22pm UTC](https://forum.ansible.com/t/how-to-pylint-and-unit-test-module-utils-code-in-my-local-module-role/32408/3 "2020-03-06T17:22:24Z")
If you convert it to a collection you can use relative imports. I am in a very similar situation to you with the PYTHONPATH manipulation for testing. [https://github.com/ansible/awx/blob/a93b1aa3395651f01b39a03a6f122d3bdad99897/Makefile#L411](https://github.com/ansible/awx/blob/a93b1aa3395651f01b39a03a6f122d3bdad99897/Makefile#L411) I put the root of the collection project (it’s in a folder, not the top-level of source control) in the python path. Technically, this means that imports are screwed up, because module\_utils will import as `import plugins.module_utils.tower` as opposed to the proper import of `import ansible_collections.awx.awx.plugins.module_utils.tower`, but the extra step to nest those folders has been a pain point. So to keep the development-test cycle fast, I knowingly use the wrong imports in tests: [https://github.com/ansible/awx/blob/a93b1aa3395651f01b39a03a6f122d3bdad99897/awx\_collection/test/awx/conftest.py#L112](https://github.com/ansible/awx/blob/a93b1aa3395651f01b39a03a6f122d3bdad99897/awx_collection/test/awx/conftest.py#L112) This doesn’t affect the functionality of the modules that import from module\_utils, because they use relative imports. [https://github.com/ansible/awx/blob/a93b1aa3395651f01b39a03a6f122d3bdad99897/awx\_collection/plugins/modules/tower\_credential.py#L221](https://github.com/ansible/awx/blob/a93b1aa3395651f01b39a03a6f122d3bdad99897/awx_collection/plugins/modules/tower_credential.py#L221) Again, I think this is a trick you only get with collections. ---
**Author:** ![Richard\_Megginson](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/richard_megginson/32/4540_2.png) [@Richard\_Megginson](https://forum.ansible.com/u/Richard_Megginson)\ **Post date:** [March 6, 2020, 6:27pm UTC](https://forum.ansible.com/t/how-to-pylint-and-unit-test-module-utils-code-in-my-local-module-role/32408/4 "2020-03-06T18:27:12Z")
> If you convert it to a collection you can use relative imports. I am in a very similar situation to you with the PYTHONPATH manipulation for testing. > > [https://github.com/ansible/awx/blob/a93b1aa3395651f01b39a03a6f122d3bdad99897/Makefile#L411](https://github.com/ansible/awx/blob/a93b1aa3395651f01b39a03a6f122d3bdad99897/Makefile#L411) > > I put the root of the collection project (it's in a folder, not the top-level of source control) in the python path. Technically, this means that imports are screwed up, because module\_utils will import as `import plugins.module_utils.tower` as opposed to the proper import of `import ansible_collections.awx.awx.plugins.module_utils.tower`, but the extra step to nest those folders has been a pain point. So to keep the development-test cycle fast, I knowingly use the wrong imports in tests: > > [https://github.com/ansible/awx/blob/a93b1aa3395651f01b39a03a6f122d3bdad99897/awx\_collection/test/awx/conftest.py#L112](https://github.com/ansible/awx/blob/a93b1aa3395651f01b39a03a6f122d3bdad99897/awx_collection/test/awx/conftest.py#L112) > > This doesn't affect the functionality of the modules that import from module\_utils, because they use relative imports. > > [https://github.com/ansible/awx/blob/a93b1aa3395651f01b39a03a6f122d3bdad99897/awx\_collection/plugins/modules/tower\_credential.py#L221](https://github.com/ansible/awx/blob/a93b1aa3395651f01b39a03a6f122d3bdad99897/awx_collection/plugins/modules/tower_credential.py#L221) > > Again, I think this is a trick you only get with collections. Good to know - we will eventually support collections, but we will have to support the current roles for a while. > ---- > If you keep the current folder structure, I wonder if instead of messing with sys.modules in the test code, you could just make the folders `ansible/module_utils/basic`, etc. in some far-off place, have those import from the path you know works, and put that in the python path too. Same as your current idea, it's very hacky but still isolated to your testing setup. I was thinking of something like that, for tox/venv testing: install ansible in the venv cp or ln myrole/module\_utils/\* inside .tox/path/to/ansible/module\_utils That way I can script a solution for all of our system roles rather than having to disable pylint checks and custom sys.modules hacking inside unit test code. ---
**Author:** ![Richard\_Megginson](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/richard_megginson/32/4540_2.png) [@Richard\_Megginson](https://forum.ansible.com/u/Richard_Megginson)\ **Post date:** [March 6, 2020, 10:07pm UTC](https://forum.ansible.com/t/how-to-pylint-and-unit-test-module-utils-code-in-my-local-module-role/32408/5 "2020-03-06T22:07:04Z")
> > If you convert it to a collection you can use relative imports. I am in a very similar situation to you with the PYTHONPATH manipulation for testing. > > > > [https://github.com/ansible/awx/blob/a93b1aa3395651f01b39a03a6f122d3bdad99897/Makefile#L411](https://github.com/ansible/awx/blob/a93b1aa3395651f01b39a03a6f122d3bdad99897/Makefile#L411) > > > > I put the root of the collection project (it's in a folder, not the top-level of source control) in the python path. Technically, this means that imports are screwed up, because module\_utils will import as `import plugins.module_utils.tower` as opposed to the proper import of `import ansible_collections.awx.awx.plugins.module_utils.tower`, but the extra step to nest those folders has been a pain point. So to keep the development-test cycle fast, I knowingly use the wrong imports in tests: > > > > [https://github.com/ansible/awx/blob/a93b1aa3395651f01b39a03a6f122d3bdad99897/awx\_collection/test/awx/conftest.py#L112](https://github.com/ansible/awx/blob/a93b1aa3395651f01b39a03a6f122d3bdad99897/awx_collection/test/awx/conftest.py#L112) > > > > This doesn't affect the functionality of the modules that import from module\_utils, because they use relative imports. > > > > [https://github.com/ansible/awx/blob/a93b1aa3395651f01b39a03a6f122d3bdad99897/awx\_collection/plugins/modules/tower\_credential.py#L221](https://github.com/ansible/awx/blob/a93b1aa3395651f01b39a03a6f122d3bdad99897/awx_collection/plugins/modules/tower_credential.py#L221) > > > > Again, I think this is a trick you only get with collections. > > Good to know - we will eventually support collections, but we will have to support the current roles for a while. > > > ---- > > If you keep the current folder structure, I wonder if instead of messing with sys.modules in the test code, you could just make the folders `ansible/module_utils/basic`, etc. in some far-off place, have those import from the path you know works, and put that in the python path too. Same as your current idea, it's very hacky but still isolated to your testing setup. > > I was thinking of something like that, for tox/venv testing: > install ansible in the venv > cp or ln myrole/module\_utils/\* inside .tox/path/to/ansible/module\_utils > > That way I can script a solution for all of our system roles rather than having to disable pylint checks and custom sys.modules hacking inside unit test code. This works: [https://github.com/linux-system-roles/template/pull/12](https://github.com/linux-system-roles/template/pull/12)