`ansible-playbook` (`ansible-core 2.17.1`) fails on target with `python 3.9`

Get Help
,
1

Hi all,

I have an Ansible target machine running Rocky Linux 8 with Python 3.9.19 installed.

[ansible@punch2 ~]$ python3 --version
Python 3.9.19
[ansible@punch2 ~]$ which python3
/usr/bin/python3
[ansible@punch2 ~]$ file $(which python3)
/usr/bin/python3: symbolic link to /etc/alternatives/python3
[ansible@punch2 ~]$ file /etc/alternatives/python3
/etc/alternatives/python3: symbolic link to /usr/bin/python3.9

On my control node I have ansible-core 2.17.1 and python 3.12.4
According to the ansible-core support matrix [1] this should be a working combination.

However, running my playbook fails with

TASK [install rsync] *********************************************************************************************************************
fatal: [punch2]: FAILED! => {"changed": false, "module_stderr": "Shared connection to punch2 closed.\r\n", "module_stdout": "\r\nTraceback (most recent call last):\r\n  File \"<stdin>\", line 12, in <module>\r\n  File \"<frozen importlib._bootstrap>\", line 971, in _find_and_load\r\n  File \"<frozen importlib._bootstrap>\", line 951, in _find_and_load_unlocked\r\n  File \"<frozen importlib._bootstrap>\", line 894, in _find_spec\r\n  File \"<frozen importlib._bootstrap_external>\", line 1157, in find_spec\r\n  File \"<frozen importlib._bootstrap_external>\", line 1131, in _get_spec\r\n  File \"<frozen importlib._bootstrap_external>\", line 1112, in _legacy_get_spec\r\n  File \"<frozen importlib._bootstrap>\", line 441, in spec_from_loader\r\n  File \"<frozen importlib._bootstrap_external>\", line 544, in spec_from_file_location\r\n  File \"/tmp/ansible_ansible.legacy.dnf_payload_7b004s9t/ansible_ansible.legacy.dnf_payload.zip/ansible/module_utils/basic.py\", line 5\r\nSyntaxError: future feature annotations is not defined\r\n", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}

The affected task is:

    - name: "install rsync"
      ansible.builtin.dnf:
        name: "rsync"
        state: "present"

I thought perhaps ansible uses an incorrect python executable on the target host, but it looks like it uses the correct one:

$ grep -i python /tmp/ansible.log 
            "SUDO_COMMAND": "/bin/sh -c echo BECOME-SUCCESS-zjjlkkwbuieqcjgtuknfnroqnampysct ; /usr/bin/python3.9 /home/ansible/.ansible/tmp/ansible-tmp-1718955443.6279042-120117-145070664808475/AnsiballZ_setup.py",
            "_": "/usr/bin/python3.9"
            "discovered_interpreter_python": "/usr/bin/python3.9",
                "SUDO_COMMAND": "/bin/sh -c echo BECOME-SUCCESS-zjjlkkwbuieqcjgtuknfnroqnampysct ; /usr/bin/python3.9 /home/ansible/.ansible/tmp/ansible-tmp-1718955443.6279042-120117-145070664808475/AnsiballZ_setup.py",
                "_": "/usr/bin/python3.9"
            "python": {
                "executable": "/usr/bin/python3.9",
                "type": "cpython",
            "python_version": "3.9.19",
            "selinux_python_present": true,
        "ansible_playbook_python": "/usr/bin/python",
        "ansible_python": {
            "executable": "/usr/bin/python3.9",
            "type": "cpython",
        "ansible_python_version": "3.9.19",
        "ansible_selinux_python_present": true,
        "discovered_interpreter_python": "/usr/bin/python3.9",

Is this the expected behaviour?

When I downgrade ansible-core on my control node to 2.16.6, the playbook runs normally.

[1] Releases and maintenance — Ansible Community Documentation

2

You could run the playbook with -vvv to get more info, also which interpreter is actually used for that task.

3

Control node:

$ ansible-playbook -vvv -i inventory -K punch2/punch2.yml --diff > /tmp/ansiblevvv.log

$ grep -i python /tmp/ansiblevvv.log
  ansible python module location = /usr/lib/python3.12/site-packages/ansible
  python version = 3.12.4 (main, Jun  7 2024, 06:33:07) [GCC 14.1.1 20240522] (/usr/bin/python)
<punch2> Attempting python interpreter discovery
<punch2> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="ansible"' -o ConnectTimeout=10 -o 'ControlPath="[...]"' punch2 '/bin/sh -c '"'"'echo PLATFORM; uname; echo FOUND; command -v '"'"'"'"'"'"'"'"'python3.12'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'python3.11'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'python3.10'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'python3.9'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'python3.8'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'python3.7'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'/usr/bin/python3'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'python3'"'"'"'"'"'"'"'"'; echo ENDFOUND && sleep 0'"'"''
<punch2> (0, b'PLATFORM\nLinux\nFOUND\n/usr/bin/python3.9\n/usr/bin/python3\n/usr/bin/python3\nENDFOUND\n', b"OpenSSH_9.7p1, OpenSSL 3.3.1 4 Jun 2024\r\ndebug1: Reading configuration data
[...]
 "ansible_selinux_python_present": true, "ansible_selinux": {"status": "enabled", "policyvers": 33, "config_mode": "permissive", "mode": "permissive", "type": "targeted"}, "ansible_env": {"LS_COLORS": "rs=0:di=38;5;33:ln=38;5;51:mh=00:pi=40;38;5;11:so=38;5;13:do=38;5;5:bd=48;5;232;38;5;11:cd=48;5;232;38;5;3:or=48;5;232;38;5;9:mi=01;05;37;41:su=48;5;196;38;5;15:sg=48;5;11;38;5;16:ca=48;5;196;38;5;226:tw=48;5;10;38;5;16:ow=48;5;10;38;5;21:st=48;5;21;38;5;15:ex=38;5;40:*.tar=38;5;9:*.tgz=38;5;9:*.arc=38;5;9:*.arj=38;5;9:*.taz=38;5;9:*.lha=38;5;9:*.lz4=38;5;9:*.lzh=38;5;9:*.lzma=38;5;9:*.tlz=38;5;9:*.txz=38;5;9:*.tzo=38;5;9:*.t7z=38;5;9:*.zip=38;5;9:*.z=38;5;9:*.dz=38;5;9:*.gz=38;5;9:*.lrz=38;5;9:*.lz=38;5;9:*.lzo=38;5;9:*.xz=38;5;9:*.zst=38;5;9:*.tzst=38;5;9:*.bz2=38;5;9:*.bz=38;5;9:*.tbz=38;5;9:*.tbz2=38;5;9:*.tz=38;5;9:*.deb=38;5;9:*.rpm=38;5;9:*.jar=38;5;9:*.war=38;5;9:*.ear=38;5;9:*.sar=38;5;9:*.rar=38;5;9:*.alz=38;5;9:*.ace=38;5;9:*.zoo=38;5;9:*.cpio=38;5;9:*.7z=38;5;9:*.rz=38;5;9:*.cab=38;5;9:*.wim=38;5;9:*.swm=38;5;9:*.dwm=38;5;9:*.esd=38;5;9:*.jpg=38;5;13:*.jpeg=38;5;13:*.mjpg=38;5;13:*.mjpeg=38;5;13:*.gif=38;5;13:*.bmp=38;5;13:*.pbm=38;5;13:*.pgm=38;5;13:*.ppm=38;5;13:*.tga=38;5;13:*.xbm=38;5;13:*.xpm=38;5;13:*.tif=38;5;13:*.tiff=38;5;13:*.png=38;5;13:*.svg=38;5;13:*.svgz=38;5;13:*.mng=38;5;13:*.pcx=38;5;13:*.mov=38;5;13:*.mpg=38;5;13:*.mpeg=38;5;13:*.m2v=38;5;13:*.mkv=38;5;13:*.webm=38;5;13:*.ogm=38;5;13:*.mp4=38;5;13:*.m4v=38;5;13:*.mp4v=38;5;13:*.vob=38;5;13:*.qt=38;5;13:*.nuv=38;5;13:*.wmv=38;5;13:*.asf=38;5;13:*.rm=38;5;13:*.rmvb=38;5;13:*.flc=38;5;13:*.avi=38;5;13:*.fli=38;5;13:*.flv=38;5;13:*.gl=38;5;13:*.dl=38;5;13:*.xcf=38;5;13:*.xwd=38;5;13:*.yuv=38;5;13:*.cgm=38;5;13:*.emf=38;5;13:*.ogv=38;5;13:*.ogx=38;5;13:*.aac=38;5;45:*.au=38;5;45:*.flac=38;5;45:*.m4a=38;5;45:*.mid=38;5;45:*.midi=38;5;45:*.mka=38;5;45:*.mp3=38;5;45:*.mpc=38;5;45:*.ogg=38;5;45:*.ra=38;5;45:*.wav=38;5;45:*.oga=38;5;45:*.opus=38;5;45:*.spx=38;5;45:*.xspf=38;5;45:", "LANG": "en_US.UTF-8", "SUDO_GID": "1000", "SUDO_COMMAND": "/bin/sh -c echo BECOME-SUCCESS-javmnmtzucrhyqcgcyucxbreqcmqcogf ; /usr/bin/python3.9 /home/ansible/.ansible/tmp/ansible-tmp-1718966149.599945-164060-177038502487216/AnsiballZ_setup.py", "USER": "root", "PWD": "/home/ansible", "HOME": "/root", "SUDO_USER": "ansible", "SUDO_UID": "1000", "MAIL": "/var/mail/root", "SHELL": "/bin/bash", "TERM": "xterm-256color", "SHLVL": "1", "LOGNAME": "root", "PATH": "/sbin:/bin:/usr/sbin:/usr/bin", "_": "/usr/bin/python3.9"}, "ansible_iscsi_iqn": "", "ansible_dns": {"search": ["gsi.de"], "nameservers": ["140.181.96.11", "140.181.96.29"]}, "ansible_local": {}, "ansible_hostnqn": "nqn.2014-08.org.nvmexpress:uuid:00000000-0000-0000-0000-000000000000", "ansible_python": {"version": {"major": 3, "minor": 9, "micro": 19, "releaselevel": "final", "serial": 0}, "version_info": [3, 9, 19, "final", 0], "executable": "/usr/bin/python3.9", "has_sslcontext": true, "type": "cpython"}, 
[...]
Using module file /usr/lib/python3.12/site-packages/ansible/modules/dnf.py
<punch2> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="ansible"' -o ConnectTimeout=10 -o 'ControlPath="[...]"' -tt punch2 '/bin/sh -c '"'"'sudo -H -S -p "[sudo via ansible, key=tabodqpozselrbxwgvbgexkvfbcgajss] password:" -u root /bin/sh -c '"'"'"'"'"'"'"'"'echo BECOME-SUCCESS-tabodqpozselrbxwgvbgexkvfbcgajss ; /usr/bin/python3.9 /home/ansible/.ansible/tmp/ansible-tmp-1718966151.9287078-164118-26674811716498/AnsiballZ_dnf.py'"'"'"'"'"'"'"'"' && sleep 0'"'"''

Trimmed the output a bit.

It appears to me it tries to use /usr/bin/python3.9.

4

Dnf is a special module that requires the Python dnf bindings to work. It uses a concept called module respawn that will respawn as the Python interpreter that the bindings it requires is installed on. Unfortunately for EL8 based hosts that will be an older Python version which is not compatible with Ansible 2.17. There’s unfortunately not much you can do about modules like this as they require those Python bindings and those bindings are only available on a limited set of Python versions.

1 Like
5

In that case I would argue that the documentation at [1] is incorrect, since it suggests any Python version between 3.7 and 3.12 on the target is sufficient.

Also I would argue that the decision to require a relatively new Python version for a task as trivial as calling an executable (in this case dnf) and parsing its output to be inappropriate.
Implementing this using Python bindings may be a bit more luxurious for the programmer of the ansible module, but at the vast expense of everyone who needs ansible to work on EL8 targets.

2 Likes