In a pickle...

Archives Ansible Project
1

The error message:
ERROR! Unexpected Exception, this is probably a bug: cannot pickle ‘module’ object

I was accessing a new collection, icinga.icinga, and it complained so I decided to update… huge mistake. Now I can’t keep track of what I have and why this error is happening. I’m guessing I need to route out the 2.9 version but wanted some advice before i go… into the valley below. :wink:

Thanks for any tips…
Chris.

The current state of things:

ansible --version

ansible 2.9.0
config file = /Users/user/Documents/code/gitlab/devops/ansible/base/ansible.cfg
configured module search path = [‘/Users/user/.ansible/plugins/modules’, ‘/usr/share/ansible/plugins/modules’]
ansible python module location = /usr/local/lib/python3.9/site-packages/ansible
executable location = /usr/local/bin/ansible
python version = 3.9.13 (main, Aug 7 2022, 01:33:23) [Clang 13.1.6 (clang-1316.0.21.2.5)]

python3 -m pip show ansible

Name: ansible
Version: 6.2.0
Summary: Radically simple IT automation
Home-page: https://ansible.com/
Author: Ansible, Inc.
Author-email: info@ansible.com
License: GPLv3+
Location: /Users/user/Library/Python/3.9/lib/python/site-packages
Requires: ansible-core
Required-by:

~/Documents/code/gitlab/devops/ansible/base | python3 -m pip show ansible-core

Name: ansible-core
Version: 2.13.2
Summary: Radically simple IT automation
Home-page: https://ansible.com/
Author: Ansible, Inc.
Author-email: info@ansible.com
License: GPLv3+
Location: /Users/user/Library/Python/3.9/lib/python/site-packages
Requires: cryptography, jinja2, packaging, PyYAML, resolvelib
Required-by: ansible

The Details:

I was running a simple ping just to test the accessibility of the hosts in my hosts file. The same happens on any hosts file…

ansible -vvv all -m ping -i hosts/myhosts.yml

ansible 2.9.0
config file = /Users/user/Documents/code/gitlab/devops/ansible/base/ansible.cfg
configured module search path = [‘/Users/user/.ansible/plugins/modules’, ‘/usr/share/ansible/plugins/modules’]
ansible python module location = /usr/local/lib/python3.9/site-packages/ansible
executable location = /usr/local/bin/ansible
python version = 3.9.13 (main, Aug 7 2022, 01:33:23) [Clang 13.1.6 (clang-1316.0.21.2.5)]
Using /Users/user/Documents/code/gitlab/devops/ansible/base/ansible.cfg as config file
host_list declined parsing /Users/user/Documents/code/gitlab/devops/ansible/base/hosts/hosts_deva.yml as it did not pass its verify_file() method
script declined parsing /Users/user/Documents/code/gitlab/devops/ansible/base/hosts/hosts_deva.yml as it did not pass its verify_file() method
Parsed /Users/user/Documents/code/gitlab/devops/ansible/base/hosts/hosts_deva.yml inventory source with yaml plugin
META: ran handlers
ERROR! Unexpected Exception, this is probably a bug: cannot pickle ‘module’ object
the full traceback was:

Traceback (most recent call last):
File “/usr/local/bin/ansible”, line 123, in
exit_code = cli.run()
File “/usr/local/lib/python3.9/site-packages/ansible/cli/adhoc.py”, line 165, in run
result = self._tqm.run(play)
File “/usr/local/lib/python3.9/site-packages/ansible/executor/task_queue_manager.py”, line 240, in run
play_return = strategy.run(iterator, play_context)
File “/usr/local/lib/python3.9/site-packages/ansible/plugins/strategy/linear.py”, line 310, in run
self._queue_task(host, task, task_vars, play_context)
File “/usr/local/lib/python3.9/site-packages/ansible/plugins/strategy/init.py”, line 343, in _queue_task
worker_prc.start()
File “/usr/local/lib/python3.9/site-packages/ansible/executor/process/worker.py”, line 96, in start
return super(WorkerProcess, self).start()
File “/usr/local/Cellar/python@3.9/3.9.13_2/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/process.py”, line 121, in start
self._popen = self._Popen(self)
File “/usr/local/Cellar/python@3.9/3.9.13_2/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/context.py”, line 224, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File “/usr/local/Cellar/python@3.9/3.9.13_2/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/context.py”, line 284, in _Popen
return Popen(process_obj)
File “/usr/local/Cellar/python@3.9/3.9.13_2/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_spawn_posix.py”, line 32, in init
super().init(process_obj)
File “/usr/local/Cellar/python@3.9/3.9.13_2/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_fork.py”, line 19, in init
self._launch(process_obj)
File “/usr/local/Cellar/python@3.9/3.9.13_2/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_spawn_posix.py”, line 47, in _launch
reduction.dump(process_obj, fp)
File “/usr/local/Cellar/python@3.9/3.9.13_2/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/reduction.py”, line 60, in dump
ForkingPickler(file, protocol).dump(obj)
TypeError: cannot pickle ‘module’ object

2

Do you have any outdated pip modules?

% pip list -o

Perhaps one/some of them need updating?

3

That bug was addressed in v2.9.1 as part of https://github.com/ansible/ansible/pull/63591

I wouldn’t suggest simply upgrading to 2.9.1. 2.9.1 was released over 2.5 years ago, and the 2.9 release is now end of life. You should consider upgrading to at least ansible-core 2.12 (ansible 5.0)

4

Thank you.

What I have, and might have been lost in too much detail, is.

ansible --version

ansible 2.9.0
ansible python module location = /usr/local/lib/python3.9/site-packages/ansible

executable location = /usr/local/bin/ansible

python3 -m pip show ansible-core

Name: ansible-core
Version: 2.13.2
Location: /Users/user/Library/Python/3.9/lib/python/site-packages

python3 -m pip show ansible

Name: ansible
Version: 6.2.0

Location: /Users/user/Library/Python/3.9/lib/python/site-packages

I thought 2.9 was gone, but it persists. How do I inform my generic call to “ansible” to reference 2.13 vs 2.9? I think I need to replace “/usr/local/bin/ansible”.

Again, thank you for the tips.

5

It looks like you need to follow the documentation from https://packaging.python.org/en/latest/tutorials/installing-packages/#installing-to-the-user-site

That URL is linked from our install guide:

You may need to perform some additional configuration before you are able to run Ansible. See the Python documentation on installing to the user site for more information.

Effectively you need to run:

python3 -m site --user-base

Add /bin on the end, and insert that into your PATH environment variable. Generally it will be $HOME/.local/bin. That new bin path should be added before /usr/local/bin

To remove the old version, you likely just need to do a python3 uninstall ansible, just ensure the paths it informs you it will remove are within /usr/local

6

Ack. Worse than I thought… Here’s what I see when executing some of the commands from the page you referenced (thank you).

On MacOS - Monterey

python -m site --user-base

/Users/user/Library/Python/2.7 <<==== ACK!

python3 --version

Python 3.9.13

python --version

Python 2.7.18

python3 -m pip --version

pip 22.2.2 from /usr/local/lib/python3.9/site-packages/pip (python 3.9)

echo $PATH (broken out to make it easier to see - I don’t recall ever managing this PATH variable)

/usr/local/sbin:
/usr/local/bin/python3.9/bin:
/usr/local/bin:
/usr/bin:
/bin:
/usr/sbin:
/sbin:
/usr/local/go/bin:
/opt/X11/bin:
/Library/Frameworks/Mono.framework/Versions/Current/Commands

When I follow the path to python in /usr/local/bin/, I see the following:
python3 → …/Cellar/python@3.9/3.9.13_2/bin/python3
python3.7 → …/…/…/Library/Frameworks/Python.framework/Versions/3.7/bin/python3.7
python3.9 → …/Cellar/python@3.9/3.9.13_2/bin/python3.9

And here:

ls -la /Users/user/Library/Python/

total 0
drwx------ 6 user staff 192 Jun 29 16:45 .
drwx------@ 104 user staff 3328 Mar 11 09:25 …
drwx------ 4 user staff 128 May 23 2019 2.7
drwx------ 4 user staff 128 Dec 2 2020 3.7
drwx------ 4 user staff 128 Aug 4 2021 3.8
drwx------ 4 user staff 128 Aug 9 2021 3.9

I think I used brew when I installed 3.9.

Also, to confirm, python3 uninstall ansible will ask before uninstalling?

Thank you,

7

The Fix - It came down to “actually” removing (deleting) the 2.9.0 version which was a hard file references as: /usr/local/bin/ansible

I then added a sym-link as follows:

/usr/local/bin/ansible → …/Cellar/ansible/6.2.0/bin/ansible

All seems back to normal.

If an application was installed with a command, it should uninstall with the same command. Perhaps there’s a purge I missed. Seems uninstall leaves some bits of flotsam…

Thanks for your help…

8

The Fix - It came down to "actually" removing (deleting) the 2.9.0 version which was a hard file references as: /usr/local/bin/ansible

I then added a sym-link as follows:

/usr/local/bin/ansible -> ../Cellar/ansible/6.2.0/bin/ansible

All seems back to normal.

If an application was installed with a command, it should uninstall with the same command. Perhaps there's a purge I missed. Seems uninstall leaves some bits of flotsam...

Thanks for your help...

What command created this hardlink?