When developing a role that has a local module with some code split into module_utils, how do you set up your pythonpath or venv so that pylint and unit testing resolves the imports correctly?
There isn’t a great answer here. roles weren’t originally designed to work this way.
You may want to look into collections instead, which have proper python imports, that aren’t effectively “fake”. A collection can include modules/plugins, as well as roles. However the location of the module moves.
This doesn’t really solve things for older versions of Ansible however, as collections are a recent addition.
I put the root of the collection project (it’s in a folder, not the top-level of source control) in the python path. Technically, this means that imports are screwed up, because module_utils will import as import plugins.module_utils.tower as opposed to the proper import of import ansible_collections.awx.awx.plugins.module_utils.tower, but the extra step to nest those folders has been a pain point. So to keep the development-test cycle fast, I knowingly use the wrong imports in tests:
I put the root of the collection project (it's in a folder, not the top-level of source control) in the python path. Technically, this means that imports are screwed up, because module_utils will import as `import plugins.module_utils.tower` as opposed to the proper import of `import ansible_collections.awx.awx.plugins.module_utils.tower`, but the extra step to nest those folders has been a pain point. So to keep the development-test cycle fast, I knowingly use the wrong imports in tests:
Again, I think this is a trick you only get with collections.
Good to know - we will eventually support collections, but we will have to support the current roles for a while.
----
If you keep the current folder structure, I wonder if instead of messing with sys.modules in the test code, you could just make the folders `ansible/module_utils/basic`, etc. in some far-off place, have those import from the path you know works, and put that in the python path too. Same as your current idea, it's very hacky but still isolated to your testing setup.
I was thinking of something like that, for tox/venv testing:
install ansible in the venv
cp or ln myrole/module_utils/* inside .tox/path/to/ansible/module_utils
That way I can script a solution for all of our system roles rather than having to disable pylint checks and custom sys.modules hacking inside unit test code.
I put the root of the collection project (it's in a folder, not the top-level of source control) in the python path. Technically, this means that imports are screwed up, because module_utils will import as `import plugins.module_utils.tower` as opposed to the proper import of `import ansible_collections.awx.awx.plugins.module_utils.tower`, but the extra step to nest those folders has been a pain point. So to keep the development-test cycle fast, I knowingly use the wrong imports in tests:
Again, I think this is a trick you only get with collections.
Good to know - we will eventually support collections, but we will have to support the current roles for a while.
----
If you keep the current folder structure, I wonder if instead of messing with sys.modules in the test code, you could just make the folders `ansible/module_utils/basic`, etc. in some far-off place, have those import from the path you know works, and put that in the python path too. Same as your current idea, it's very hacky but still isolated to your testing setup.
I was thinking of something like that, for tox/venv testing:
install ansible in the venv
cp or ln myrole/module_utils/* inside .tox/path/to/ansible/module_utils
That way I can script a solution for all of our system roles rather than having to disable pylint checks and custom sys.modules hacking inside unit test code.