# Error in loading plugin in unit tests: "expected str, bytes or os.PathLike object, not NoneType"

**URL:** <https://forum.ansible.com/t/error-in-loading-plugin-in-unit-tests-expected-str-bytes-or-os-pathlike-object-not-nonetype/3246>\
**Category:** Get Help\
**Created:** [January 10, 2024, 5:30pm UTC](https://forum.ansible.com/t/error-in-loading-plugin-in-unit-tests-expected-str-bytes-or-os-pathlike-object-not-nonetype/3246 "2024-01-10T17:30:16Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![emx](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/emx/32/1961_2.png) [@emx](https://forum.ansible.com/u/emx)\
**Post date:** [January 10, 2024, 5:30pm UTC](https://forum.ansible.com/t/error-in-loading-plugin-in-unit-tests-expected-str-bytes-or-os-pathlike-object-not-nonetype/3246/1 "2024-01-10T17:30:16Z")

</div>

We have custom lookup plugin, which internally loads the “dig” plugin, like this:

```auto
from ansible.plugins.loader import lookup_loader
dig_module = lookup_loader.get("community.general.dig")

```

We also have unit tests (using the Python `unittest` module) that exercise this lookup plugin directly, outside of Ansible.

This all worked fine under Ansible 2.14. Under 2.16.2 it still works when run as part of an Ansible playbook, but in the unit tests the `lookup_loader.get()` returns None and prints the following warning:

```auto
[WARNING]: errors were encountered during the plugin load for community.general.dig: ['expected str, bytes or os.PathLike object, not NoneType']

```

This can be reproduced just by running the above two lines in a Python shell. (We’re using Python 3.12.1.)

I’m guessing that some extra initialization code is now necessary, which Ansible runs when it starts, so we need to replicate that in our unit tests, but what? I’m not even sure how to get a stack trace to find where exactly this error is coming from.

---

<div class="post-metadata">

**Author:** ![felixfontein](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/felixfontein/32/7530_2.png) [@felixfontein](https://forum.ansible.com/u/felixfontein)\
**Post date:** [January 11, 2024, 6:16am UTC](https://forum.ansible.com/t/error-in-loading-plugin-in-unit-tests-expected-str-bytes-or-os-pathlike-object-not-nonetype/3246/2 "2024-01-11T06:16:14Z")

</div>

I don’t get that warning, but I can reproduce the problem.

Do you run the unit tests with `ansible-test`? Several of the lookup plugin unit tests in community.general (and likely other collections) use `lookup_loader.get()`, and these still pass.

I guess the problem is that you are not running your tests with `ansible-test`, and thus outside of Ansible. This has never been supported, so that it did work in the past is not a promise that it will still work now.

I would suggest you use ansibe-test to run the unit tests (and organize the unit tests similar to how they are organized in other collections so ansible-test is happy with them).

---

<div class="post-metadata">

**Author:** ![emx](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/emx/32/1961_2.png) [@emx](https://forum.ansible.com/u/emx)\
**Post date:** [January 11, 2024, 10:25am UTC](https://forum.ansible.com/t/error-in-loading-plugin-in-unit-tests-expected-str-bytes-or-os-pathlike-object-not-nonetype/3246/3 "2024-01-11T10:25:08Z")

</div>

Correct, we’re not using ansible-test.

I tried to get that working, but without success. We don’t have a “collection” for our plugin - it’s just in the lookup\_plugins directory under the playbook directory, and I really don’t want to make any major changes to this just to fix one little unit test.

First it failed with “FATAL: The current working directory must be within the source tree being tested.” which lead me to [Support running ansible-test on collections outside a collection root · Issue #60215 · ansible/ansible · GitHub](https://github.com/ansible/ansible/issues/60215) - a lot of discussion there, but no action to fix this.

Finally I managed to fool it by renaming our ansible directory to “ansible\_collections” and creating a “fakenamespace/fakecollection/tests” directory.

Then it still failed with “FATAL: Command “git ls-files -z --cached --others --exclude-standard” returned exit status 128.” I don’t understand why git is involved here at all. OK, I worked around that. Installed pytest. It still fails:

```auto
$ ansible-test units -v
Configured locale: en_US.UTF-8
WARNING: Skipping unit tests on Python 2.7 because it is only supported by module/module_utils unit tests. No module/module_utils unit tests were selected.
WARNING: Skipping unit tests on Python 3.8 because it is only supported by module/module_utils unit tests. No module/module_utils unit tests were selected.
WARNING: Skipping unit tests on Python 3.9 because it is only supported by module/module_utils unit tests. No module/module_utils unit tests were selected.
Creating container database.
WARNING: Skipping unit tests on Python 3.10 because it could not be found.
WARNING: Skipping unit tests on Python 3.11 because it could not be found.
Unit test controller with Python 3.12
Initializing "/tmp/ansible-test-uj5wyphx-injector" as the temporary injector directory.
Injecting "/tmp/python-dsobpbb8-ansible/python" as a execv wrapper for the "/home/em/.direnv/python-3.12.1/bin/python" interpreter.
Stream command: pytest -r a -n auto --color yes -p no:cacheprovider -c /home/em/.direnv/python-3.12.1/lib/python3.12/site-packages/ansible_test/_data/py ...
ERROR: usage: __main__.py [options] [file_or_dir] [file_or_dir] [...]
__main__.py: error: unrecognized arguments: -n tests/unit/test_netplan.py tests/unit/test_plugins.py
  inifile: /home/em/.direnv/python-3.12.1/lib/python3.12/site-packages/ansible_test/_data/pytest/config/default.ini
  rootdir: /media/sf_work/lops2/ansible_collections/fakenamespace/fakecollection

FATAL: Command "pytest -r a -n auto --color yes -p no:cacheprovider -c /home/em/.direnv/python-3.12.1/lib/python3.12/site-packages/ansible_test/_data/pytest/config/default.ini --junit-xml /media/sf_work/lops2/ansible_collections/fakenamespace/fakecollection/tests/output/junit/python3.12-controller-units.xml --strict-markers --rootdir /media/sf_work/lops2/ansible_collections/fakenamespace/fakecollection -v tests/unit/test_netplan.py tests/unit/test_plugins.py" returned exit status 4.

```

So I’ve given up on ansible-test now. I’m sure there is just some one-line tweak I need to add to work around the `expected str, bytes or os.PathLike object, not NoneType` warning. Yes, I know it’s not the supported way, but it seems that the supported way is very, very awkward for my (and apparently many other people’s) use case.

---

<div class="post-metadata">

**Author:** ![emx](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/emx/32/1961_2.png) [@emx](https://forum.ansible.com/u/emx)\
**Post date:** [January 11, 2024, 11:05am UTC](https://forum.ansible.com/t/error-in-loading-plugin-in-unit-tests-expected-str-bytes-or-os-pathlike-object-not-nonetype/3246/4 "2024-01-11T11:05:50Z")

</div>

I dug into the code a bit more to see what it’s doing:

```auto
>>> lookup_loader._find_fq_plugin("community.general.dig", ".py", plugin_load_context=PluginLoadContext())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/em/.direnv/python-3.12.1/lib/python3.12/site-packages/ansible/plugins/loader.py", line 552, in _find_fq_plugin
    pkg_path = os.path.dirname(pkg. __file__ )
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen posixpath>", line 181, in dirname
TypeError: expected str, bytes or os.PathLike object, not NoneType

```

So now I have a stack trace. This error happens because

`sys.modules.get('ansible_collections.community.general.plugins.lookup'). __file__ is None`

Adding an empty ` __init__.py` file to the directory returned by `. __path__ ` for that module (“~/.direnv/python-3.12.1/lib/python3.12/site-packages/ansible\_collections/community/general/plugins/lookup” in my case) works around the problem and the unit test runs.

It’s quite a silly thing to have to do, but hey, it works! Maybe manipulating `sys.path` in the unit test would be a better workaround - I’m not sure.

I’m not sure if that [code](https://github.com/ansible/ansible/blob/devel/lib/ansible/plugins/loader.py#L551) could just use `pkg. __path__ ` instead of `os.path.dirname(pkg. __file__ )`. They’re the same on in my case, but I don’t know enough Python to say if they’d always be the same.

---

<div class="post-metadata">

**Author:** ![felixfontein](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/felixfontein/32/7530_2.png) [@felixfontein](https://forum.ansible.com/u/felixfontein)\
**Post date:** [January 11, 2024, 11:52am UTC](https://forum.ansible.com/t/error-in-loading-plugin-in-unit-tests-expected-str-bytes-or-os-pathlike-object-not-nonetype/3246/5 "2024-01-11T11:52:03Z")

</div>

> [@emx](#):
>
> We don’t have a “collection” for our plugin - it’s just in the lookup\_plugins directory under the playbook directory, and I really don’t want to make any major changes to this just to fix one little unit test.

ansible-test is for collections (and ansible-core), so if you don’t have one, you cannot use it.

I would really suggest to move your plugin into a collection (you can have a local collection `foo.bar` local to your playbooks in a `collections/ansible_collections/foo/bar` subdirectory, you don’t need to keep it in a separate repository or even build and publish it).

> [@emx](#):
>
> So now I have a stack trace. This error happens because
> 
> `sys.modules.get('ansible_collections.community.general.plugins.lookup'). __file__ is None`
> 
> Adding an empty ` __init__.py` file to the directory returned by `. __path__ ` for that module (“~/.direnv/python-3.12.1/lib/python3.12/site-packages/ansible\_collections/community/general/plugins/lookup” in my case) works around the problem and the unit test runs.

That’s because you are not using ansible-core’s Python module (as in: Python module, not Ansible module) loader, which doesn’t need the ` __init__.py` files to be present, but you’re (implicitly) using the plain Python module loader. I guess for older versions that one was added magically when using the plugin loader, but now it isn’t.

You should probably run `init_plugin_loader()` from `ansible.plugins.loader` before using the loader, that seems to initialize everything:

```auto
from ansible.plugins.loader import lookup_loader, init_plugin_loader
init_plugin_loader()
dig_module = lookup_loader.get("community.general.dig")

```

---

<div class="post-metadata">

**Author:** ![emx](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/emx/32/1961_2.png) [@emx](https://forum.ansible.com/u/emx)\
**Post date:** [January 11, 2024, 12:52pm UTC](https://forum.ansible.com/t/error-in-loading-plugin-in-unit-tests-expected-str-bytes-or-os-pathlike-object-not-nonetype/3246/6 "2024-01-11T12:52:40Z")

</div>

That works, thank you so much!

If I run that in our filter plugin I get a warning:

> /home/em/.direnv/python-3.12.1/lib/python3.12/site-packages/ansible/plugins/loader.py:1479: UserWarning: AnsibleCollectionFinder has already been configured  
> warnings.warn(‘AnsibleCollectionFinder has already been configured’)

so I added `init_plugin_loader()` to the unit test only, and now it passes without the ` __init__.py` file.

---

<div class="post-metadata">

**Author:** ![felixfontein](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/felixfontein/32/7530_2.png) [@felixfontein](https://forum.ansible.com/u/felixfontein)\
**Post date:** [January 11, 2024, 7:26pm UTC](https://forum.ansible.com/t/error-in-loading-plugin-in-unit-tests-expected-str-bytes-or-os-pathlike-object-not-nonetype/3246/7 "2024-01-11T19:26:04Z")

</div>

ansible-core already sets up the loaders, otherwise it couldn’t have loaded your plugin. So only call that function from the unit tests 🙂

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/ansible/original/1X/1882d9a01da5d8996a862359b0913cc568a39b84.svg) [@system](https://forum.ansible.com/u/system)\
**Post date:** [February 10, 2024, 7:26pm UTC](https://forum.ansible.com/t/error-in-loading-plugin-in-unit-tests-expected-str-bytes-or-os-pathlike-object-not-nonetype/3246/8 "2024-02-10T19:26:40Z")

</div>

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.