yum/dnf/package modules hanging on whole system update

Archives Ansible Project
,
1

I’m trying to perform a full system update with the yum module but ansible just hangs for a little bit more than an hour before failing.

Here is the command I’m using:

ansible all -u node-user -b --become-user=root -i exec/inventory -m yum -a ‘name=* state=latest’ -vvvv --limit=worker1

Here is the output (redacted):

ansible [core 2.12.5]
config file = /home/nicolas/test-upgrade-os/ansible.cfg
configured module search path = [‘/home/nicolas/.ansible/plugins/modules’, ‘/usr/share/ansible/plugins/modules’]
ansible python module location = /home/nicolas/test-upgrade-os/config/venv/lib64/python3.8/site-packages/ansible
ansible collection location = /home/nicolas/.ansible/collections:/usr/share/ansible/collections
executable location = ./config/venv/bin/ansible
python version = 3.8.16 (default, Jun 25 2023, 05:53:51) [GCC 8.5.0 20210514 (Red Hat 8.5.0-18)]
jinja version = 3.1.2
libyaml = True
Using /home/nicolas/test-upgrade-os/ansible.cfg as config file
setting up inventory plugins
host_list declined parsing /home/nicolas/test-upgrade-os/exec/inventory as it did not pass its verify_file() method
script declined parsing /home/nicolas/test-upgrade-os/exec/inventory as it did not pass its verify_file() method
auto declined parsing /home/nicolas/test-upgrade-os/exec/inventory as it did not pass its verify_file() method
Parsed /home/nicolas/test-upgrade-os/exec/inventory inventory source with ini plugin
Loading callback plugin minimal of type stdout, v2.0 from /home/nicolas/test-upgrade-os/config/venv/lib64/python3.8/site-packages/ansible/plugins/callback/minimal.py
Skipping callback ‘default’, as we already have a stdout callback.
Skipping callback ‘minimal’, as we already have a stdout callback.
Skipping callback ‘oneline’, as we already have a stdout callback.
META: ran handlers
<10.10.0.101> ESTABLISH SSH CONNECTION FOR USER: node-user
<10.10.0.101> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ‘User=“node-user”’ -o ConnectTimeout=10 -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/node-user -o ‘ProxyCommand=ssh -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/bastion-user -W %h:%p -p22 bastion-user@W.X.Y.Z’ -o ‘ControlPath=“/home/nicolas/test-upgrade-os/config/ansible/cp/09896940d7”’ 10.10.0.101 ‘/bin/sh -c ‘"’“‘echo ~node-user && sleep 0’”’“‘’
<10.10.0.101> (0, b’/home/node-user\n’, b’')
<10.10.0.101> ESTABLISH SSH CONNECTION FOR USER: node-user
<10.10.0.101> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ‘User=“node-user”’ -o ConnectTimeout=10 -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/node-user -o ‘ProxyCommand=ssh -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/bastion-user -W %h:%p -p22 bastion-user@W.X.Y.Z’ -o 'ControlPath=”/home/nicolas/test-upgrade-os/config/ansible/cp/09896940d7"’ 10.10.0.101 ‘/bin/sh -c ‘"’"’( umask 77 && mkdir -p “echo /home/node-user/.ansible/tmp”&& mkdir “echo /home/node-user/.ansible/tmp/ansible-tmp-1691583637.8116903-3768362-148267575047576” && echo ansible-tmp-1691583637.8116903-3768362-148267575047576=“echo /home/node-user/.ansible/tmp/ansible-tmp-1691583637.8116903-3768362-148267575047576” ) && sleep 0’“'”‘’
<10.10.0.101> (0, b’ansible-tmp-1691583637.8116903-3768362-148267575047576=/home/node-user/.ansible/tmp/ansible-tmp-1691583637.8116903-3768362-148267575047576\n’, b’‘)
Attempting python interpreter discovery
<10.10.0.101> ESTABLISH SSH CONNECTION FOR USER: node-user
<10.10.0.101> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ‘User=“node-user”’ -o ConnectTimeout=10 -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/node-user -o ‘ProxyCommand=ssh -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/bastion-user -W %h:%p -p22 bastion-user@W.X.Y.Z’ -o ‘ControlPath=“/home/nicolas/test-upgrade-os/config/ansible/cp/09896940d7”’ 10.10.0.101 ‘/bin/sh -c ‘"’“'echo PLATFORM; uname; echo FOUND; command -v '”’"’“'”‘"’“'”‘“‘python3.10’”’“'”‘"’“'”‘"’“'; command -v '”‘"’“'”‘"’“'”‘“‘python3.9’”’“'”‘"’“'”‘"’“'; command -v '”‘"’“'”‘"’“'”‘“‘python3.8’”’“'”‘"’“'”‘"’“'; command -v '”‘"’“'”‘"’“'”‘“‘python3.7’”’“'”‘"’“'”‘"’“'; command -v '”‘"’“'”‘"’“'”‘“‘python3.6’”’“'”‘"’“'”‘"’“'; command -v '”‘"’“'”‘"’“'”‘“‘python3.5’”’“'”‘"’“'”‘"’“'; command -v '”‘"’“'”‘"’“'”‘"’/usr/bin/python3’“'”‘"’“'”‘"’“'”‘; command -v ‘"’"’“'”‘"’“'”‘"’/usr/libexec/platform-python’“'”‘"’“'”‘"’“'”‘; command -v ‘"’"’“'”‘"’“'”‘“‘python2.7’”’“'”‘"’“'”‘"’“'; command -v '”‘"’“'”‘"’“'”‘“‘python2.6’”’“'”‘"’“'”‘"’“'; command -v '”‘"’“'”‘"’“'”‘"’/usr/bin/python’“'”‘"’“'”‘"’“'”‘; command -v ‘"’"’“'”‘"’“'”‘“‘python’”’“'”‘"’“'”‘"’“‘; echo ENDFOUND && sleep 0’”‘"’’
<10.10.0.101> (0, b’PLATFORM\nLinux\nFOUND\n/usr/libexec/platform-python\nENDFOUND\n’, b’‘)
<10.10.0.101> ESTABLISH SSH CONNECTION FOR USER: node-user
<10.10.0.101> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ‘User=“node-user”’ -o ConnectTimeout=10 -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/node-user -o ‘ProxyCommand=ssh -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/bastion-user -W %h:%p -p22 bastion-user@W.X.Y.Z’ -o ‘ControlPath=“/home/nicolas/test-upgrade-os/config/ansible/cp/09896940d7”’ 10.10.0.101 ‘/bin/sh -c ‘"’"’/usr/libexec/platform-python && sleep 0’“'”‘’
<10.10.0.101> (0, b’{“platform_dist_result”: [“centos”, “8.5”, “Green Obsidian”], “osrelease_content”: “NAME=\“Rocky Linux\”\nVERSION=\“8.5 (Green Obsidian)\”\nID=\“rocky\”\nID_LIKE=\“rhel centos fedora\”\nVERSION_ID=\“8.5\”\nPLATFORM_ID=\“platform:el8\”\nPRETTY_NAME=\“Rocky Linux 8.5 (Green Obsidian)\”\nANSI_COLOR=\“0;32\”\nCPE_NAME=\“cpe:/o:rocky:rocky:8:GA\”\nHOME_URL=\“https://rockylinux.org/\\"\\nBUG_REPORT_URL=\\“https://bugs.rockylinux.org/\\”\\nROCKY_SUPPORT_PRODUCT=\\"Rocky Linux\”\nROCKY_SUPPORT_PRODUCT_VERSION=\“8\”\n”}\n’, b’‘)
Using module file /home/nicolas/test-upgrade-os/config/venv/lib64/python3.8/site-packages/ansible/modules/setup.py
<10.10.0.101> PUT /home/nicolas/test-upgrade-os/config/ansible/tmp/ansible-local-3768356wtqis0tq/tmpy4qpsqz0 TO /home/node-user/.ansible/tmp/ansible-tmp-1691583637.8116903-3768362-148267575047576/AnsiballZ_setup.py
<10.10.0.101> SSH: EXEC sftp -b - -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ‘User=“k
ubonode”’ -o ConnectTimeout=10 -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/node-user -o ‘ProxyCommand=ssh -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/bastion-user -W %h:%p -p22 bastion-user@W.X.Y.Z’ -o ‘ControlPath=“/home/nicolas/test-upgrade-os/config/ansible/cp/09896940d7”’ ‘[10.10.0.101]’
<10.10.0.101> (0, b’sftp> put /home/nicolas/test-upgrade-os/config/ansible/tmp/ansible-local-3768356wtqis0tq/tmpy4qpsqz0 /home/node-user/.ansible/tmp/ansible-tmp-1691583637.8116903-3768362-148267575047576/AnsiballZ_setup.py\n’, b’‘)
<10.10.0.101> ESTABLISH SSH CONNECTION FOR USER: node-user
<10.10.0.101> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ‘User=“node-user”’ -o ConnectTimeout=10 -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/node-user -o ‘ProxyCommand=ssh -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/bastion-user -W %h:%p -p22 bastion-user@W.X.Y.Z’ -o ‘ControlPath=“/home/nicolas/test-upgrade-os/config/ansible/cp/09896940d7”’ 10.10.0.101 ‘/bin/sh -c ‘"’“‘chmod u+x /home/node-user/.ansible/tmp/ansible-tmp-1691583637.8116903-3768362-148267575047576/ /home/node-user/.ansible/tmp/ansible-tmp-1691583637.8116903-3768362-148267575047576/AnsiballZ_setup.py && sleep 0’”’"’’
<10.10.0.101> (0, b’‘, b’‘)
<10.10.0.101> ESTABLISH SSH CONNECTION FOR USER: node-user
<10.10.0.101> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ‘User=“node-user”’ -o ConnectTimeout=10 -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/node-user -o ‘ProxyCommand=ssh -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/bastion-user -W %h:%p -p22 bastion-user@W.X.Y.Z’ -o ‘ControlPath=“/home/nicolas/test-upgrade-os/config/ansible/cp/09896940d7”’ -tt 10.10.0.101 ‘/bin/sh -c ‘"’“'sudo -H -S -n -u root /bin/sh -c '”’"’“'”‘"’“'”‘“‘echo BECOME-SUCCESS-ztvxikfxzuzwogfymzcnlpfaroxhooqg ; /usr/libexec/platform-python /home/node-user/.ansible/tmp/ansible-tmp-1691583637.8116903-3768362-148267575047576/AnsiballZ_setup.py’”’“'”‘"’“'”‘"’“’ && sleep 0’”‘"’’
Escalation succeeded
<10.10.0.101> (0, b’\r\n{“ansible_facts”: {“ansible_pkg_mgr”: “dnf”}, “invocation”: {“module_args”: {“filter”: [“ansible_pkg_mgr”], “gather_subset”: [“!all”], “gather_timeout”: 10, “fact_path”: “/etc/ansible/facts.d”}}}\r\n’, b’‘)
Running ansible.legacy.dnf as the backend for the yum action plugin
Using module file /home/nicolas/test-upgrade-os/config/venv/lib64/python3.8/site-packages/ansible/modules/dnf.py
<10.10.0.101> PUT /home/nicolas/test-upgrade-os/config/ansible/tmp/ansible-local-3768356wtqis0tq/tmpomw666d5 TO /home/node-user/.ansible/tmp/ansible-tmp-1691583637.8116903-3768362-148267575047576/AnsiballZ_dnf.py
<10.10.0.101> SSH: EXEC sftp -b - -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ‘User=“node-user”’ -o ConnectTimeout=10 -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/node-user -o ‘ProxyCommand=ssh -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/bastion-user -W %h:%p -p22 bastion-user@W.X.Y.Z’ -o ‘ControlPath=“/home/nicolas/test-upgrade-os/config/ansible/cp/09896940d7”’ ‘[10.10.0.101]’
<10.10.0.101> (0, b’sftp> put /home/nicolas/test-upgrade-os/config/ansible/tmp/ansible-local-3768356wtqis0tq/tmpomw666d5 /home/node-user/.ansible/tmp/ansible-tmp-1691583637.8116903-3768362-148267575047576/AnsiballZ_dnf.py\n’, b
‘’)
<10.10.0.101> ESTABLISH SSH CONNECTION FOR USER: node-user
<10.10.0.101> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ‘User=“node-user”’ -o ConnectTimeout=10 -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/node-user -o ‘ProxyCommand=ssh -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/bastion-user -W %h:%p -p22 bastion-user@W.X.Y.Z’ -o ‘ControlPath=“/home/nicolas/test-upgrade-os/config/ansible/cp/09896940d7”’ 10.10.0.101 ‘/bin/sh -c ‘"’“‘chmod u+x /home/node-user/.ansible/tmp/ansible-tmp-1691583637.8116903-3768362-148267575047576/ /home/node-user/.ansible/tmp/ansible-tmp-1691583637.8116903-3768362-148267575047576/AnsiballZ_dnf.py && sleep 0’”’“‘’
<10.10.0.101> (0, b’‘, b’')
<10.10.0.101> ESTABLISH SSH CONNECTION FOR USER: node-user
<10.10.0.101> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ‘User=“node-user”’ -o ConnectTimeout=10 -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/node-user -o ‘ProxyCommand=ssh -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/bastion-user -W %h:%p -p22 bastion-user@W.X.Y.Z’ -o 'ControlPath=”/home/nicolas/test-upgrade-os/config/ansible/cp/09896940d7"’ -tt 10.10.0.101 ‘/bin/sh -c ‘"’“'sudo -H -S -n -u root /bin/sh -c '”’“'”‘"’“'”‘"’“‘echo BECOME-SUCCESS-gjdfwphkqonajiudmalgairdspobkjad ; /usr/libexec/platform-python /home/node-user/.ansible/tmp/ansible-tmp-1691583637.8116903-3768362-148267575047576/AnsiballZ_dnf.py’”‘"’“'”‘"’“'”‘"’ && sleep 0’“'”‘’
Escalation succeeded

Before running ansible, I ssh’ed in the node and ran:

watch “ps -aux | grep ansible”

While ansible was performing the yum update, I saw the process /usr/libexec/platform-python /home/node-user/.ansible/tmp/ansible-tmp-1691583637.8116903-3768362-148267575047576/AnsiballZ_dnf.py was running for about 10-15 minutes and when it had disappear, ansible kept running for more than an hour before failing with the following error:

worker1 | UNREACHABLE! => {
“changed”: false,
“msg”: "Failed to connect to the host via ssh: ",
“unreachable”: true
}

I tried using the dnf and package modules, which gave the exact same results.

I tried updating a single package (tar) and it worked with yum, dnf and package modules.

I’m running ansible on a Rocky Linux 8 machine with python 3.8.16. The worker1 machine is also using Rocky Linux 8 and the output of /usr/libexec/platform-python --version is Python 3.6.8.

Should I file an issue in the ansible github repo for this matter ? Or am I doing something wrong ?

2

Check the host and see what happens on a full manual update. I have had issues with ansible when the yum command was hanging on a host do to a local issue with updating. Single packages were fine, but a full host update failed. I had to resolve the full update issue on the host.

3

Thanks for stepping in to help.

I did run sudo yum update -y directly in one of my hosts, and everything went well.

Also, I created the following playbook and surprisingly it works:

  • hosts: all
    gather_facts: no
    tasks:
  • name: YUM | Get available package updates
    yum:
    list: updates
    register: yum_available_package_updates
  • name: YUM | Update packages
    yum:
    name: “{{ yum_available_package_updates.results | map(attribute=‘name’) | list }}”
    state: ‘latest’
    register: yum_upgrade
  • name: YUM | Reboot after packages updates
    when:
  • yum_upgrade.changed
    reboot:

However, if I use it as an ansible role, like so:

4

A little fantasy, but nice playbook. Have you checked you transaction history for the failed task. If there is a time out or permission issue for the ansible user it might show up there. Another idea given your yaml is to stick debug messages at each step to see what the output is between stages.

5

Looking at your verbose output, it looks like your ansible runs are tunneled through a bastion/jumphost/proxy?

When you run your “yum update” directly on a host, are you doing:

[user@control-node ~]$ ssh user@target
[user@target-node ~] sudo yum -y update

Or are you doing:

[user@control-node ~]$ ssh user@target-node sudo yum -y update

I’m just wondering if there’s something unusual in the bastion connection handling, or the shell environment of a full interactive shell with a TTY vs. an ansible run?

Similarly, you have your -vvv output of a failing run. If you do -vvv for a working run, does that cast any light, indicate any differences in behaviour in connection, privilege escalation or command invocation?

Do you have any proxies defined that may be being picked up from the environment in an interactive session which aren’t in an ansible run?

6

Good catch on the jumphost Will. If that is timing out mid patch cycle due to duration of yum upgrade job you would get this behavior.

7

@Evan
Here is the output of “yum history” after running the playbook that uses the yum update task as a role:
ID | Command line | Date and time | Action(s) | Altered

8

I sent a message 13 hours ago which never was posted to the thread… I don’t know why, but I hope it will be approved before this one ! I’ll repost the details from the previous message if it doesn’t get posted here.

Here is the output of the working run in verbose mode (redacted, again):<10.10.0.101> ESTABLISH SSH CONNECTION FOR USER: node-user
<10.10.0.101> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ‘User=“node-user”’ -o ConnectTimeout=10 -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/node-user -o ‘ProxyCommand=ssh -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/bastion-user -W %h:%p -p22 bastion@W.X.Y.Z’ -o ‘ControlPath=“/home/nicolas/test-upgrade-os/config/ansible/cp/35923d7a2f”’ 10.10.0.101 ‘/bin/sh -c ‘"’“‘echo ~node-user && sleep 0’”’“‘’
<10.10.0.101> (0, b’/home/node-user\n’, b’')
<10.10.0.101> ESTABLISH SSH CONNECTION FOR USER: node-user
<10.10.0.101> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ‘User=“node-user”’ -o ConnectTimeout=10 -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/node-user -o ‘ProxyCommand=ssh -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/bastion-user -W %h:%p -p22 bastion@W.X.Y.Z’ -o 'ControlPath=”/home/nicolas/test-upgrade-os/config/ansible/cp/35923d7a2f"’ 10.10.0.101 ‘/bin/sh -c ‘"’"’( umask 77 && mkdir -p “echo /home/node-user/.ansible/tmp”&& mkdir “echo /home/node-user/.ansible/tmp/ansible-tmp-1691623057.6383471-3972386-257445419832497” && echo ansible-tmp-1691623057.6383471-3972386-257445419832497=“echo /home/node-user/.ansible/tmp/ansible-tmp-1691623057.6383471-3972386-257445419832497” ) && sleep 0’“'”‘’
<10.10.0.101> (0, b’ansible-tmp-1691623057.6383471-3972386-257445419832497=/home/node-user/.ansible/tmp/ansible-tmp-1691623057.6383471-3972386-257445419832497\n’, b’‘)
Using module file /home/nicolas/test-upgrade-os/config/venv/lib64/python3.8/site-packages/ansible/modules/setup.py
<10.10.0.101> PUT /home/nicolas/test-upgrade-os/config/ansible/tmp/ansible-local-3972316to208f1n/tmp72j3do1e TO /home/node-user/.ansible/tmp/ansible-tmp-1691623057.6383471-3972386-257445419832497/AnsiballZ_setup.py
<10.10.0.101> SSH: EXEC sftp -b - -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ‘User=“node-user”’ -o ConnectTimeout=10 -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/node-user -o ‘ProxyCommand=ssh -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/bastion-user -W %h:%p -p22 bastion@W.X.Y.Z’ -o ‘ControlPath=“/home/nicolas/test-upgrade-os/config/ansible/cp/35923d7a2f”’ ‘[10.10.0.101]’
<10.10.0.101> (0, b’sftp> put /home/nicolas/test-upgrade-os/config/ansible/tmp/ansible-local-3972316to208f1n/tmp72j3do1e /home/node-user/.ansible/tmp/ansible-tmp-1691623057.6383471-3972386-257445419832497/AnsiballZ_setup.py\n’,b’‘)
<10.10.0.101> ESTABLISH SSH CONNECTION FOR USER: node-user
<10.10.0.101> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ‘User=“node-user”’ -o ConnectTimeout=10 -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/node-user -o ‘ProxyCommand=ssh -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/bastion-user -W %h:%p -p22 bastion@W.X.Y.Z’ -o ‘ControlPath=“/home/nicolas/test-upgrade-os/config/ansible/cp/35923d7a2f”’ 10.10.0.101 ‘/bin/sh -c ‘"’“‘chmod u+x /home/node-user/.ansible/tmp/ansible-tmp-1691623057.6383471-3972386-257445419832497/ /home/node-user/.ansible/tmp/ansible-tmp-1691623057.6383471-3972386-257445419832497/AnsiballZ_setup.py && sleep 0’”’"’’
<10.10.0.101> (0, b’‘, b’‘)
<10.10.0.101> ESTABLISH SSH CONNECTION FOR USER: node-user
<10.10.0.101> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ‘User=“node-user”’ -o ConnectTimeout=10 -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/node-user -o ‘ProxyCommand=ssh -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/bastion-user -W %h:%p -p22 bastion@W.X.Y.Z’ -o ‘ControlPath=“/home/nicolas/test-upgrade-os/config/ansible/cp/35923d7a2f”’ -tt 10.10.0.101 ‘/bin/sh -c ‘"’“'sudo -H -S -n -u root /bin/sh -c '”’"’“'”‘"’“'”‘“‘echo BECOME-SUCCESS-xyjrtllhapjdxgzmdeyucwfptikmhufx ; /usr/libexec/platform-python /home/node-user/.ansible/tmp/ansible-tmp-1691623057.6383471-3972386-257445419832497/AnsiballZ_setup.py’”’“'”‘"’“'”‘"’“’ && sleep 0’”‘"’’
Escalation succeeded
<10.10.0.101> (0, b’\r\n{“ansible_facts”: {“ansible_pkg_mgr”: “dnf”}, “invocation”: {“module_args”: {“filter”: [“ansible_pkg_mgr”], “gather_subset”: [“!all”], “gather_timeout”: 10, “fact_path”: “/etc/ansible/facts.d”}}}\r\n’, b’‘)
Running ansible.legacy.dnf as the backend for the yum action plugin
Using module file /home/nicolas/test-upgrade-os/config/venv/lib64/python3.8/site-packages/ansible/modules/dnf.py
<10.10.0.101> PUT /home/nicolas/test-upgrade-os/config/ansible/tmp/ansible-local-3972316to208f1n/tmpnkohamzi TO /home/node-user/.ansible/tmp/ansible-tmp-1691623057.6383471-3972386-257445419832497/AnsiballZ_dnf.py
<10.10.0.101> SSH: EXEC sftp -b - -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ‘User=“node-user”’ -o ConnectTimeout=10 -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/node-user -o ‘ProxyCommand=ssh -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/bastion-user -W %h:%p -p22 bastion@W.X.Y.Z’ -o ‘ControlPath=“/home/nicolas/test-upgrade-os/config/ansible/cp/35923d7a2f”’ ‘[10.10.0.101]’
<10.10.0.101> (0, b’sftp> put /home/nicolas/test-upgrade-os/config/ansible/tmp/ansible-local-3972316to208f1n/tmpnkohamzi /home/node-user/.ansible/tmp/ansible-tmp-1691623057.6383471-3972386-257445419832497/AnsiballZ_dnf.py\n’, b’‘)
<10.10.0.101> ESTABLISH SSH CONNECTION FOR USER: node-user
<10.10.0.101> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ‘User=“node-user”’ -o ConnectTimeout=10 -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/node-user -o ‘ProxyCommand=ssh -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/bastion-user -W %h:%p -p22 bastion@W.X.Y.Z’ -o ‘ControlPath=“/home/nicolas/test-upgrade-os/config/ansible/cp/35923d7a2f”’ 10.10.0.101 ‘/bin/sh -c ‘"’“‘chmod u+x /home/node-user/.ansible/tmp/ansible-tmp-1691623057.6383471-3972386-257445419832497/ /home/node-user/.ansible/tmp/ansible-tmp-1691623057.6383471-3972386-257445419832497/AnsiballZ_dnf.py && sleep 0’”’"’’
<10.10.0.101> (0, b’‘, b’‘)
<10.10.0.101> ESTABLISH SSH CONNECTION FOR USER: node-user
<10.10.0.101> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ‘User=“node-user”’ -o ConnectTimeout=10 -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/node-user -o ‘ProxyCommand=ssh -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/bastion-user -W %h:%p -p22 bastion@W.X.Y.Z’ -o ‘ControlPath=“/home/nicolas/test-upgrade-os/config/ansible/cp/35923d7a2f”’ -tt 10.10.0.101 ‘/bin/sh -c ‘"’“'sudo -H -S -n -u root /bin/sh -c '”’"’“'”‘"’“'”‘“‘echo BECOME-SUCCESS-khbomwysuyxjpfmwseamqcfildpbpwnl ; /usr/libexec/platform-python /home/node-user/.ansible/tmp/ansible-tmp-1691623057.6383471-3972386-257445419832497/AnsiballZ_dnf.py’”’“'”‘"’“'”‘"’“’ && sleep 0’”‘"’’
Escalation succeeded
<10.10.0.101> (0, b’\r\n{“msg”: “”, “changed”: true, “results”: [“Installed: libgomp-8.5.0-18.el8.x86_64”, “Installed: util-linux-2.32.1-42.el8_8.x86_64”, “Installed: rocky-logos-86.3-1.el8.x86_64”, “Installed: vim-minimal-2:8.0.1763-19.el8_6.4.x86_64”, “Installed: virt-what-1.25-3.el8.x86_64”, “Installed: python3-netifaces-0.10.6-4.el8.x86_64”, “Installed: which-2.21-18.el8.x86_64”, “Installed: xfsprogs-5.0.0-11.el8_8.x86_64”, “Installed: xz-5.2.4-4.el8_6.x86_64”, “Installed: libini_config-1.3.1-40.el8.x86_64”, “Installed: xz-libs-5.2.4-4.el8_6.x86_64”, “Installed: yum-4.7.0-16.el8_8.noarch”, “Installed: yum-utils-4.0.21-19.el8_8.noarch”, “Installed: zlib-1.2.11-21.el8_7.x86_64”, “Installed: libksba-1.3.5-9.el8_7.x86_64”, “Installed: libldb-2.6.1-1.el8.x86_64”, “Installed: platform-python-3.6.8-51.el8_8.1.rocky.0.x86_64”, “Installed: platform-python-pip-9.0.3-22.el8.rocky.0.noarch”, “Installed: platform-python-setuptools-39.2.0-7.el8.noarch”, “Installed: policycoreutils-2.9-24.el8.x86_64”, “Installed: libmount-2.32.1-42.el8_8.x86_64”, “Installed: libestr-0.1.10-3.el8.x86_64”, “Installed: policycoreutils-python-utils-2.9-24.el8.noarch”, “Installed: NetworkManager-1:1.40.16-4.el8_8.x86_64”, “Installed: polkit-0.115-15.el8.x86_64”, “Installed: polkit-libs-0.115-15.el8.x86_64”, “Installed: pixman-0.38.4-2.el8.x86_64”, “Installed: libnfsidmap-1:2.3.3-59.el8.x86_64”, “Installed: procps-ng-3.3.15-13.el8.x86_64”, “Installed: NetworkManager-initscripts-updown-1:1.40.16-4.el8_8.noarch”, “Installed: libnl3-3.7.0-1.el8.x86_64”, “Installed: python3-libxml2-2.9.7-16.el8_8.1.x86_64”, “Installed: libnl3-cli-3.7.0-1.el8.x86_64”, “Installed: NetworkManager-libnm-1:1.40.16-4.el8_8.x86_64”, “Installed: python3-audit-3.0.7-4.el8.x86_64”, “Installed: python3-linux-procfs-0.7.1-1.el8.noarch”, “Installed: libpath_utils-0.2.1-40.el8.x86_64”, “Installed: python3-unbound-1.16.2-5.el8.x86_64”, “Installed: NetworkManager-team-1:1.40.16-4.el8_8.x86_64”, “Installed: NetworkManager-tui-1:1.40.16-4.el8_8.x86_64”, “Installed: python3-dnf-4.7.0-16.el8_8.noarch”, “Installed: python3-dnf-plugins-core-4.0.21-19.el8_8.noarch”, “Installed: python3-gpg-1.13.1-11.el8.x86_64”, “Installed: python3-hawkey-0.63.0-14.el8_8.x86_64”, “Installed: libpwquality-1.4.4-6.el8.x86_64”, “Installed: python3-libcomps-0.1.18-1.el8.x86_64”, “Installed: python3-libdnf-0.63.0-14.el8_8.x86_64”, “Installed: python3-libs-3.6.8-51.el8_8.1.rocky.0.x86_64”, “Installed: python3-libselinux-2.9-8.el8.x86_64”, “Installed: python3-libsemanage-2.9-9.el8_6.x86_64”, “Installed: libref_array-0.1.5-40.el8.x86_64”, “Installed: librepo-1.14.2-4.el8.x86_64”, “Installed: audit-3.0.7-4.el8.x86_64”, “Installed: gnupg2-2.2.20-3.el8_6.x86_64”, “Installed: audit-libs-3.0.7-4.el8.x86_64”, “Installed: gnupg2-smime-2.2.20-3.el8_6.x86_64”, “Installed: gnutls-3.6.16-6.el8_7.x86_64”, “Installed: python3-magic-5.33-24.el8.noarch”, “Installed: libreport-filesystem-2.9.5-15.el8.rocky.6.3.x86_64”, “Installed: libseccomp-2.5.2-1.el8.x86_64”, “Installed: authselect-1.2.6-1.el8.x86_64”, “Installed: gpgme-1.13.1-11.el8.x86_64”, “Installed: authselect-libs-1.2.6-1.el8.x86_64”, “Installed: libsecret-0.18.6-1.el8.0.2.x86_64”, “Installed: libselinux-2.9-8.el8.x86_64”, “Installed: libselinux-utils-2.9-8.el8.x86_64”, “Installed: grub2-common-1:2.02-148.el8_8.1.rocky.0.3.noarch”, “Installed: libsemanage-2.9-9.el8_6.x86_64”, “Installed: python3-perf-4.18.0-477.21.1.el8_8.x86_64”, “Installed: python3-pip-wheel-9.0.3-22.el8.rocky.0.noarch”, “Installed: python3-policycoreutils-2.9-24.el8.noarch”, “Installed: bash-4.4.20-4.el8_6.x86_64”, “Installed: libsmartcols-2.32.1-42.el8_8.x86_64”, “Installed: python3-rpm-4.14.3-26.el8.x86_64”, “Installed: authselect-compat-1.2.6-1.el8.x86_64”, “Installed: grub2-pc-1:2.02-148.el8_8.1.rocky.0.3.x86_64”, “Installed: grub2-pc-modules-1:2.02-148.el8_8.1.rocky.0.3.noarch”, “Installed: libsolv-0.7.20-4.el8_7.x86_64”, “Installed: bind-export-libs-32:9.11.36-8.el8_8.1.x86_64”, “Installed: grub2-tools-1:2.02-148.el8_8.1.rocky.0.3.x86_64”, “Installed: grub2-tools-efi-1:2.02-148.el8_8.1.rocky.0.3.x86_64”, “Installed: libsoup-2.62.3-3.el8.x86_64”, “Installed: python3-setuptools-39.2.0-7.el8.noarch”, “Installed: grub2-tools-extra-1:2.02-148.el8_8.1.rocky.0.3.x86_64”, “Installed: grub2-tools-minimal-1:2.02-148.el8_8.1.rocky.0.3.x86_64”, “Installed: grubby-8.40-47.el8.x86_64”, “Installed: libss-1.45.6-5.el8.x86_64”, “Installed: gssproxy-0.8.0-21.el8.x86_64”, “Installed: gzip-1.9-13.el8_5.x86_64”, “Installed: libssh-0.9.6-10.el8_8.x86_64”, “Installed: libssh-config-0.9.6-10.el8_8.noarch”, “Installed: libsss_autofs-2.8.2-3.el8_8.x86_64”, “Installed: hwdata-0.314-8.16.el8.noarch”, “Installed: libsss_certmap-2.8.2-3.el8_8.x86_64”, “Installed: python3-slip-0.6.4-13.el8.noarch”, “Installed: python3-slip-dbus-0.6.4-13.el8.noarch”, “Installed: python3-syspurpose-1.28.36-2.el8.x86_64”, “Installed: libsss_idmap-2.8.2-3.el8_8.x86_64”, “Installed: libsss_nss_idmap-2.8.2-3.el8_8.x86_64”, “Installed: info-6.5-7.el8.x86_64”, “Installed: initscripts-10.00.18-1.el8.x86_64”, “Installed: c-ares-1.13.0-6.el8_8.2.x86_64”, “Installed: libsss_sudo-2.8.2-3.el8_8.x86_64”, “Installed: libstdc+±8.5.0-18.el8.x86_64”, “Installed: ca-certificates-2022.2.54-80.2.el8_6.noarch”, “Installed: iproute-5.18.0-1.1.el8_8.x86_64”, “Installed: chrony-4.2-1.el8.rocky.1.0.x86_64”, “Installed: cockpit-bridge-286.1-1.el8.x86_64”, “Installed: libsysfs-2.1.0-25.el8.x86_64”, “Installed: cockpit-system-286.1-1.el8.noarch”, “Installed: cockpit-ws-286.1-1.el8.x86_64”, “Installed: libtalloc-2.3.4-1.el8.x86_64”, “Installed: rng-tools-6.15-3.el8.x86_64”, “Installed: coreutils-8.30-15.el8.x86_64”, “Installed: coreutils-common-8.30-15.el8.x86_64”, “Installed: libtasn1-4.13-4.el8_7.x86_64”, “Installed: cpio-2.12-11.el8.x86_64”, “Installed: libtdb-1.4.7-1.el8.x86_64”, “Installed: unbound-libs-1.16.2-5.el8.x86_64”, “Installed: iputils-20180629-10.el8.x86_64”, “Installed: libteam-1.31-4.el8.x86_64”, “Installed: cronie-1.5.2-8.el8.x86_64”, “Installed: cronie-anacron-1.5.2-8.el8.x86_64”, “Installed: irqbalance-2:1.9.0-4.el8.x86_64”, “Installed: libtevent-0.13.0-1.el8.x86_64”, “Installed: crypto-policies-20221215-1.gitece0092.el8.noarch”, “Installed: crypto-policies-scripts-20221215-1.gitece0092.el8.noarch”, “Installed: libtirpc-1.1.4-8.el8.x86_64”, “Installed: rocky-release-8.8-1.8.el8.noarch”, “Installed: cryptsetup-libs-2.3.7-5.el8.x86_64”, “Installed: rocky-repos-8.8-1.8.el8.noarch”, “Installed: rpcbind-1.2.5-10.el8.x86_64”, “Installed: rpm-4.14.3-26.el8.x86_64”, “Installed: rpm-build-libs-4.14.3-26.el8.x86_64”, “Installed: rpm-libs-4.14.3-26.el8.x86_64”, “Installed: curl-7.61.1-30.el8_8.3.x86_64”, “Installed: rpm-plugin-selinux-4.14.3-26.el8.x86_64”, “Installed: rpm-plugin-systemd-inhibit-4.14.3-26.el8.x86_64”, “Installed: qemu-guest-agent-15:6.2.0-32.module+el8.8.0+1279+230c2115.x86_64”, “Installed: rsync-3.1.3-19.el8_7.1.x86_64”, “Installed: python3-setools-4.3.0-3.el8.x86_64”, “Installed: libuser-0.62-25.el8.x86_64”, “Installed: jansson-2.14-1.el8.x86_64”, “Installed: libuuid-2.32.1-42.el8_8.x86_64”, “Installed: json-c-0.13.1-3.el8.x86_64”, “Installed: python3-setuptools-wheel-39.2.0-7.el8.noarch”, “Installed: dbus-1:1.12.8-24.el8_8.1.x86_64”, “Installed: libverto-0.3.2-2.el8.x86_64”, “Installed: dbus-common-1:1.12.8-24.el8_8.1.noarch”, “Installed: dbus-daemon-1:1.12.8-24.el8_8.1.x86_64”, “Installed: kernel-4.18.0-477.21.1.el8_8.x86_64”, “Installed: libverto-libevent-0.3.2-2.el8.x86_64”, “Installed: dbus-libs-1:1.12.8-24.el8_8.1.x86_64”, “Installed: dbus-tools-1:1.12.8-24.el8_8.1.x86_64”, “Installed: kernel-core-4.18.0-477.21.1.el8_8.x86_64”, “Installed: libxml2-2.9.7-16.el8_8.1.x86_64”, “Installed: linux-firmware-20230404-114.git2e92a49f.el8_8.noarch”, “Installed: dhcp-client-12:4.3.6-49.el8.x86_64”, “Installed: dhcp-common-12:4.3.6-49.el8.noarch”, “Installed: dhcp-libs-12:4.3.6-49.el8.x86_64”, “Installed: sed-4.5-5.el8.x86_64”, “Installed: dmidecode-1:3.3-4.el8.x86_64”, “Installed: dnf-4.7.0-16.el8_8.noarch”, “Installed: selinux-policy-3.14.3-117.el8_8.2.noarch”, “Installed: dnf-data-4.7.0-16.el8_8.noarch”, “Installed: lmdb-libs-0.9.24-2.el8.x86_64”, “Installed: dnf-plugins-core-4.0.21-19.el8_8.noarch”, “Installed: tzdata-2023c-1.el8.noarch”, “Installed: oddjob-0.34.7-3.el8.x86_64”, “Installed: oddjob-mkhomedir-0.34.7-3.el8.x86_64”, “Installed: dracut-049-223.git20230119.el8.x86_64”, “Installed: dracut-config-generic-049-223.git20230119.el8.x86_64”, “Installed: dracut-config-rescue-049-223.git20230119.el8.x86_64”, “Installed: logrotate-3.14.0-6.el8.x86_64”, “Installed: setroubleshoot-server-3.3.26-5.el8.x86_64”, “Installed: dracut-network-049-223.git20230119.el8.x86_64”, “Installed: dracut-squash-049-223.git20230119.el8.x86_64”, “Installed: kernel-modules-4.18.0-477.21.1.el8_8.x86_64”, “Installed: e2fsprogs-1.45.6-5.el8.x86_64”, “Installed: selinux-policy-targeted-3.14.3-117.el8_8.2.noarch”, “Installed: setup-2.12.2-9.el8.noarch”, “Installed: sg3_utils-1.44-6.el8.x86_64”, “Installed: e2fsprogs-libs-1.45.6-5.el8.x86_64”, “Installed: sg3_utils-libs-1.44-6.el8.x86_64”, “Installed: shadow-utils-2:4.6-17.el8.x86_64”, “Installed: kernel-tools-4.18.0-477.21.1.el8_8.x86_64”, “Installed: kernel-tools-libs-4.18.0-477.21.1.el8_8.x86_64”, “Installed: kexec-tools-2.0.25-5.el8_8.1.x86_64”, “Installed: elfutils-debuginfod-client-0.188-3.el8.x86_64”, “Installed: elfutils-default-yama-scope-0.188-3.el8.noarch”, “Installed: kmod-25-19.el8.x86_64”, “Installed: sos-4.5.5-2.el8.noarch”, “Installed: elfutils-libelf-0.188-3.el8.x86_64”, “Installed: kmod-libs-25-19.el8.x86_64”, “Installed: kpartx-0.8.4-37.el8.x86_64”, “Installed: elfutils-libs-0.188-3.el8.x86_64”, “Installed: memstrack-0.2.4-2.el8.x86_64”, “Installed: ethtool-2:5.13-2.el8.x86_64”, “Installed: sqlite-3.26.0-18.el8_8.x86_64”, “Installed: expat-2.2.5-11.el8.x86_64”, “Installed: microcode_ctl-4:20220809-2.20230214.1.el8_8.x86_64”, “Installed: krb5-libs-1.18.2-25.el8_8.x86_64”, “Installed: file-5.33-24.el8.x86_64”, “Installed: file-libs-5.33-24.el8.x86_64”, “Installed: sqlite-libs-3.26.0-18.el8_8.x86_64”, “Installed: sssd-client-2.8.2-3.el8_8.x86_64”, “Installed: sssd-common-2.8.2-3.el8_8.x86_64”, “Installed: freetype-2.9.1-9.el8.x86_64”,“Installed: sssd-kcm-2.8.2-3.el8_8.x86_64”, “Installed: fuse-libs-2.9.7-16.el8.x86_64”, “Installed: libarchive-3.3.3-5.el8.x86_64”, “Installed: sssd-nfs-idmap-2.8.2-3.el8_8.x86_64”, “Installed: nfs-utils-1:2.3.3-59.el8.x86_64”, “Installed: gawk-4.2.1-4.el8.x86_64”, “Installed: gdbm-1:1.18-2.el8.x86_64”, “Installed: libbasicobjects-0.1.1-40.el8.x86_64”, “Installed: libblkid-2.32.1-42.el8_8.x86_64”, “Installed: gdbm-libs-1:1.18-2.el8.x86_64”, “Installed: gdisk-1.0.3-11.el8.x86_64”, “Installed: cairo-1.15.12-6.el8.x86_64”, “Installed: libbpf-0.5.0-1.el8.x86_64”, “Installed: sudo-1.8.29-10.el8.x86_64”, “Installed: cairo-gobject-1.15.12-6.el8.x86_64”, “Installed: libcap-2.48-5.el8_8.x86_64”, “Installed: sscg-3.0.0-7.el8.x86_64”, “Installed: systemd-239-74.el8_8.3.x86_64”, “Installed: glib2-2.56.4-161.el8.x86_64”, “Installed: libcollection-0.7.0-40.el8.x86_64”, “Installed: libcom_err-1.45.6-5.el8.x86_64”, “Installed: glibc-2.28-225.el8.x86_64”, “Installed: glibc-all-langpacks-2.28-225.el8.x86_64”, “Installed: glibc-common-2.28-225.el8.x86_64”, “Installed: libcomps-0.1.18-1.el8.x86_64”, “Installed: systemd-libs-239-74.el8_8.3.x86_64”, “Installed: glibc-gconv-extra-2.28-225.el8.x86_64”, “Installed: openssh-8.0p1-19.el8_8.x86_64”, “Installed: systemd-pam-239-74.el8_8.3.x86_64”, “Installed: libcurl-7.61.1-30.el8_8.3.x86_64”, “Installed: openssh-clients-8.0p1-19.el8_8.x86_64”, “Installed: systemd-udev-239-74.el8_8.3.x86_64”, “Installed: openssh-server-8.0p1-19.el8_8.x86_64”, “Installed: openssl-1:1.1.1k-9.el8_7.x86_64”, “Installed: tar-2:1.30-9.el8.x86_64”, “Installed: openssl-libs-1:1.1.1k-9.el8_7.x86_64”, “Installed: libdhash-0.5.0-40.el8.x86_64”, “Installed: openssl-pkcs11-0.4.10-3.el8.x86_64”, “Installed: teamd-1.31-4.el8.x86_64”, “Installed: libdnf-0.63.0-14.el8_8.x86_64”, “Installed: cloud-init-22.1-8.el8_8.1.0.1.noarch”, “Installed: cloud-utils-growpart-0.33-0.el8.noarch”, “Installed: pam-1.3.1-25.el8.x86_64”, “Installed: PackageKit-1.1.12-6.el8.0.2.x86_64”, “Installed: passwd-0.80-4.el8.x86_64”, “Installed: PackageKit-glib-1.1.12-6.el8.0.2.x86_64”, “Installed: libfdisk-2.32.1-42.el8_8.x86_64”, “Installed: pciutils-libs-3.7.0-3.el8.x86_64”, “Installed: libffi-3.1-24.el8.x86_64”, “Installed: tuned-2.20.0-1.el8.noarch”, “Installed: libgcc-8.5.0-18.el8.x86_64”, “Installed: rsyslog-8.2102.0-13.el8.x86_64”, “Installed: libgcrypt-1.8.5-7.el8_6.x86_64”, “Installed: rocky-gpg-keys-8.8-1.8.el8.noarch”, “Installed: pcre2-10.32-3.el8_6.x86_64”, “Removed: NetworkManager-1:1.32.10-4.el8.x86_64”, “Removed: NetworkManager-libnm-1:1.32.10-4.el8.x86_64”, “Removed: NetworkManager-team-1:1.32.10-4.el8.x86_64”, “Removed: NetworkManager-tui-1:1.32.10-4.el8.x86_64”, “Removed: PackageKit-1.1.12-6.el8.x86_64”, “Removed: PackageKit-glib-1.1.12-6.el8.x86_64”, “Removed: audit-3.0-0.17.20191104git1c2f876.el8.1.x86_64”, “Removed: audit-libs-3.0-0.17.20191104git1c2f876.el8.1.x86_64”, “Removed: authselect-1.2.2-3.el8.x86_64”, “Removed: authselect-compat-1.2.2-3.el8.x86_64”, “Removed: authselect-libs-1.2.2-3.el8.x86_64”, “Removed: bash-4.4.20-2.el8.x86_64”, “Removed: bind-export-libs-32:9.11.26-6.el8.x86_64”, “Removed: c-ares-1.13.0-5.el8.x86_64”, “Removed: ca-certificates-2021.2.50-80.0.el8_4.noarch”, “Removed: cairo-1.15.12-3.el8.x86_64”, “Removed: cairo-gobject-1.15.12-3.el8.x86_64”, “Removed: chrony-4.1-1.el8.x86_64”, “Removed: cloud-init-21.1-7.el8_5.3.noarch”, “Removed: cloud-utils-growpart-0.31-3.el8.noarch”, “Removed: cockpit-bridge-251.3-1.el8_5.x86_64”, “Removed: cockpit-system-251.3-1.el8_5.noarch”, “Removed: cockpit-ws-251.3-1.el8_5.x86_64”, “Removed: coreutils-8.30-12.el8.x86_64”, “Removed: coreutils-common-8.30-12.el8.x86_64”, “Removed: cpio-2.12-10.el8.x86_64”, “Removed: cronie-1.5.2-4.el8.x86_64”, “Removed: cronie-anacron-1.5.2-4.el8.x86_64”, “Removed: crypto-policies-20210617-1.gitc776d3e.el8.noarch”, “Removed: crypto-policies-scripts-20210617-1.gitc776d3e.el8.noarch”, “Removed: cryptsetup-libs-2.3.3-4.el8_5.1.x86_64”, “Removed: curl-7.61.1-22.el8.x86_64”, “Removed: dbus-1:1.12.8-14.el8.x86_64”, “Removed: dbus-common-1:1.12.8-14.el8.noarch”, “Removed: dbus-daemon-1:1.12.8-14.el8.x86_64”, “Removed: dbus-libs-1:1.12.8-14.el8.x86_64”, “Removed: dbus-tools-1:1.12.8-14.el8.x86_64”, “Removed: dhcp-client-12:4.3.6-45.el8.x86_64”,“Removed: dhcp-common-12:4.3.6-45.el8.noarch”, “Removed: dhcp-libs-12:4.3.6-45.el8.x86_64”, “Removed: dmidecode-1:3.2-10.el8.x86_64”, “Removed: dnf-4.7.0-4.el8.noarch”, “Removed: dnf-data-4.7.0-4.el8.noarch”, “Removed: dnf-plugins-core-4.0.21-4.el8_5.noarch”, “Removed: dracut-049-191.git20210920.el8.x86_64”, “Removed: dracut-config-generic-049-191.git20210920.el8.x86_64”, “Removed: dracut-config-rescue-049-191.git20210920.el8.x86_64”, “Removed: dracut-network-049-191.git20210920.el8.x86_64”, “Removed: dracut-squash-049-191.git20210920.el8.x86_64”, “Removed: e2fsprogs-1.45.6-2.el8.x86_64”, “Removed: e2fsprogs-libs-1.45.6-2.el8.x86_64”, “Removed: elfutils-debuginfod-client-0.185-1.el8.x86_64”, “Removed: elfutils-default-yama-scope-0.185-1.el8.noarch”, “Removed: elfutils-libelf-0.185-1.el8.x86_64”, “Removed: elfutils-libs-0.185-1.el8.x86_64”, “Removed: ethtool-2:5.8-7.el8.x86_64”, “Removed: expat-2.2.5-4.el8.x86_64”, “Removed: file-5.33-20.el8.x86_64”, “Removed: file-libs-5.33-20.el8.x86_64”, “Removed: freetype-2.9.1-4.el8_3.1.x86_64”, “Removed: fuse-libs-2.9.7-12.el8.x86_64”, “Removed: gawk-4.2.1-2.el8.x86_64”, “Removed: gdbm-1:1.18-1.el8.x86_64”, “Removed: gdbm-libs-1:1.18-1.el8.x86_64”, “Removed: gdisk-1.0.3-6.el8.x86_64”, “Removed: glib2-2.56.4-156.el8.x86_64”, “Removed: glibc-2.28-164.el8.x86_64”, “Removed: glibc-all-langpacks-2.28-164.el8.x86_64”, “Removed: glibc-common-2.28-164.el8.x86_64”, “Removed:gnupg2-2.2.20-2.el8.x86_64”, “Removed: gnupg2-smime-2.2.20-2.el8.x86_64”, “Removed: gnutls-3.6.16-4.el8.x86_64”, “Removed: gpgme-1.13.1-9.el8.x86_64”, “Removed: grub2-common-1:2.02-106.el8.0.2.noarch”, “Removed: grub2-pc-1:2.02-106.el8.0.2.x86_64”, “Removed: grub2-pc-modules-1:2.02-106.el8.0.2.noarch”, “Removed: grub2-tools-1:2.02-106.el8.0.2.x86_64”, “Removed: grub2-tools-extra-1:2.02-106.el8.0.2.x86_64”, “Removed: grub2-tools-minimal-1:2.02-106.el8.0.2.x86_64”, “Removed: grubby-8.40-42.el8.x86_64”, “Removed: gssproxy-0.8.0-19.el8.x86_64”, “Removed: gzip-1.9-12.el8.x86_64”, “Removed: hwdata-0.314-8.10.el8.noarch”, “Removed: info-6.5-6.el8.x86_64”, “Removed: initscripts-10.00.15-1.el8.x86_64”, “Removed: iproute-5.12.0-4.el8.x86_64”, “Removed: iputils-20180629-7.el8.x86_64”, “Removed: irqbalance-2:1.4.0-6.el8.x86_64”, “Removed: jansson-2.11-3.el8.x86_64”, “Removed: json-c-0.13.1-2.el8.x86_64”, “Removed: kernel-tools-4.18.0-348.12.2.el8_5.x86_64”, “Removed: kernel-tools-libs-4.18.0-348.12.2.el8_5.x86_64”, “Removed: kexec-tools-2.0.20-57.el8_5.1.x86_64”, “Removed: kmod-25-18.el8.x86_64”, “Removed: kmod-libs-25-18.el8.x86_64”, “Removed: kpartx-0.8.4-17.el8.x86_64”, “Removed:krb5-libs-1.18.2-14.el8.x86_64”, “Removed: libarchive-3.3.3-1.el8_5.x86_64”, “Removed: libbasicobjects-0.1.1-39.el8.x86_64”, “Removed: libblkid-2.32.1-28.el8.x86_64”, “Removed: libbpf-0.4.0-1.el8.x86_64”, “Removed: libcap-2.26-5.el8.x86_64”, “Removed: libcollection-0.7.0-39.el8.x86_64”, “Removed: libcom_err-1.45.6-2.el8.x86_64”, “Removed: libcomps-0.1.16-2.el8.x86_64”, “Removed: libcurl-7.61.1-22.el8.x86_64”, “Removed: libdhash-0.5.0-39.el8.x86_64”, “Removed: libdnf-0.63.0-3.el8.x86_64”, “Removed: libestr-0.1.10-1.el8.x86_64”, “Removed: libfdisk-2.32.1-28.el8.x86_64”, “Removed: libffi-3.1-22.el8.x86_64”, “Removed: libgcc-8.5.0-4.el8_5.x86_64”, “Removed: libgcrypt-1.8.5-6.el8.x86_64”, “Removed: libgomp-8.5.0-4.el8_5.x86_64”, “Removed: libini_config-1.3.1-39.el8.x86_64”, “Removed: libksba-1.3.5-7.el8.x86_64”, “Removed: libldb-2.3.0-2.el8.x86_64”, “Removed: libmount-2.32.1-28.el8.x86_64”, “Removed: libnfsidmap-1:2.3.3-46.el8.x86_64”, “Removed: libnl3-3.5.0-1.el8.x86_64”, “Removed: libnl3-cli-3.5.0-1.el8.x86_64”, “Removed: libpath_utils-0.2.1-39.el8.x86_64”, “Removed: libpwquality-1.4.4-3.el8.x86_64”, “Removed: libref_array-0.1.5-39.el8.x86_64”, “Removed: librepo-1.14.0-2.el8.x86_64”, “Removed: libreport-filesystem-2.9.5-15.el8.rocky.2.x86_64”, “Removed: libseccomp-2.5.1-1.el8.x86_64”, “Removed: libsecret-0.18.6-1.el8.x86_64”, “Removed: libselinux-2.9-5.el8.x86_64”, “Removed: libselinux-utils-2.9-5.el8.x86_64”, “Removed: libsemanage-2.9-6.el8.x86_64”, “Removed: libsmartcols-2.32.1-28.el8.x86_64”, “Removed: libsolv-0.7.19-1.el8.x86_64”, “Removed: libsoup-2.62.3-2.el8.x86_64”, “Removed: libss-1.45.6-2.el8.x86_64”, “Removed: libssh-0.9.4-3.el8.x86_64”, “Removed: libssh-config-0.9.4-3.el8.noarch”, “Removed: libsss_autofs-2.5.2-2.el8_5.4.x86_64”, “Removed: libsss_certmap-2.5.2-2.el8_5.4.x86_64”, “Removed: libsss_idmap-2.5.2-2.el8_5.4.x86_64”, “Removed: libsss_nss_idmap-2.5.2-2.el8_5.4.x86_64”, “Removed: libsss_sudo-2.5.2-2.el8_5.4.x86_64”, “Removed: libstdc+±8.5.0-4.el8_5.x86_64”, “Removed: libsysfs-2.1.0-24.el8.x86_64”, “Removed: libtalloc-2.3.2-1.el8.x86_64”, “Removed: libtasn1-4.13-3.el8.x86_64”, “Removed: libtdb-1.4.3-1.el8.x86_64”, “Removed: libteam-1.31-2.el8.x86_64”, “Removed: libtevent-0.11.0-0.el8.x86_64”, “Removed: libtirpc-1.1.4-5.el8.x86_64”, “Removed: libuser-0.62-23.el8.x86_64”, “Removed: libuuid-2.32.1-28.el8.x86_64”, “Removed: libverto-0.3.0-5.el8.x86_64”, “Removed: libverto-libevent-0.3.0-5.el8.x86_64”, “Removed: libxml2-2.9.7-11.el8.x86_64”, “Removed: linux-firmware-20210702-103.gitd79c2677.el8.noarch”, “Removed: lmdb-libs-0.9.24-1.el8.x86_64”, “Removed: logrotate-3.14.0-4.el8.x86_64”, “Removed: memstrack-0.1.11-1.el8.x86_64”, “Removed: microcode_ctl-4:20210608-1.el8.x86_64”, “Removed: nfs-utils-1:2.3.3-46.el8.x86_64”, “Removed: oddjob-0.34.7-1.el8.x86_64”, “Removed: oddjob-mkhomedir-0.34.7-1.el8.x86_64”, “Removed: openssh-8.0p1-10.el8.x86_64”, “Removed: openssh-clients-8.0p1-10.el8.x86_64”, “Removed: openssh-server-8.0p1-10.el8.x86_64”, “Removed: openssl-1:1.1.1k-5.el8_5.x86_64”, “Removed: openssl-libs-1:1.1.1k-5.el8_5.x86_64”, “Removed: openssl-pkcs11-0.4.10-2.el8.x86_64”, “Removed: pam-1.3.1-15.el8.x86_64”, “Removed: passwd-0.80-3.el8.x86_64”, “Removed: pciutils-libs-3.7.0-1.el8.x86_64”, “Removed: pcre2-10.32-2.el8.x86_64”, “Removed: pixman-0.38.4-1.el8.x86_64”, “Removed: platform-python-3.6.8-41.el8.rocky.0.x86_64”, “Removed: platform-python-pip-9.0.3-20.el8.rocky.0.noarch”, “Removed: platform-python-setuptools-39.2.0-6.el8.noarch”, “Removed: policycoreutils-2.9-16.el8.x86_64”, “Removed: policycoreutils-python-utils-2.9-16.el8.noarch”, “Removed: polkit-0.115-13.el8_5.1.x86_64”, “Removed: polkit-libs-0.115-13.el8_5.1.x86_64”, “Removed: procps-ng-3.3.15-6.el8.x86_64”, “Removed: python3-audit-3.0-0.17.20191104git1c2f876.el8.1.x86_64”, “Removed: python3-dnf-4.7.0-4.el8.noarch”, “Removed: python3-dnf-plugins-core-4.0.21-4.el8_5.noarch”, “Removed: python3-gpg-1.13.1-9.el8.x86_64”, “Removed: python3-hawkey-0.63.0-3.el8.x86_64”, “Removed: python3-libcomps-0.1.16-2.el8.x86_64”, “Removed: python3-libdnf-0.63.0-3.el8.x86_64”, “Removed: python3-libs-3.6.8-41.el8.rocky.0.x86_64”, “Removed: python3-libselinux-2.9-5.el8.x86_64”, “Removed: python3-libsemanage-2.9-6.el8.x86_64”, “Removed: python3-libxml2-2.9.7-11.el8.x86_64”, “Removed: python3-linux-procfs-0.6.3-1.el8.noarch”, “Removed: python3-perf-4.18.0-348.12.2.el8_5.x86_64”, “Removed: python3-pip-wheel-9.0.3-20.el8.rocky.0.noarch”, “Removed: python3-policycoreutils-2.9-16.el8.noarch”, “Removed: python3-rpm-4.14.3-19.el8_5.2.x86_64”, “Removed: python3-setools-4.3.0-2.el8.x86_64”, “Removed: python3-setuptools-wheel-39.2.0-6.el8.noarch”, “Removed: python3-slip-0.6.4-11.el8.noarch”, “Removed: python3-slip-dbus-0.6.4-11.el8.noarch”, “Removed: python3-syspurpose-1.28.21-3.el8.x86_64”, “Removed: python3-unbound-1.7.3-17.el8.x86_64”, “Removed: qemu-guest-agent-15:4.2.0-59.module+el8.5.0+726+ce09ee88.1.x86_64”, “Removed: rng-tools-6.13-1.git.d207e0b6.el8.x86_64”, “Removed: rocky-gpg-keys-8.5-3.el8.noarch”, “Removed: rocky-logos-85.0-3.el8.x86_64”, “Removed: rocky-release-8.5-3.el8.noarch”, “Removed: rocky-repos-8.5-3.el8.noarch”, “Removed: rpcbind-1.2.5-8.el8.x86_64”, “Removed: rpm-4.14.3-19.el8_5.2.x86_64”, “Removed: rpm-build-libs-4.14.3-19.el8_5.2.x86_64”, “Removed: rpm-libs-4.14.3-19.el8_5.2.x86_64”, “Removed: rpm-plugin-selinux-4.14.3-19.el8_5.2.x86_64”, “Removed: rpm-plugin-systemd-inhibit-4.14.3-19.el8_5.2.x86_64”, “Removed: rsync-3.1.3-12.el8.x86_64”, “Removed: rsyslog-8.2102.0-5.el8.x86_64”, “Removed: sed-4.5-2.el8.x86_64”, “Removed: selinux-policy-3.14.3-80.el8_5.2.noarch”, “Removed: selinux-policy-targeted-3.14.3-80.el8_5.2.noarch”, “Removed: setroubleshoot-server-3.3.24-4.el8.x86_64”, “Removed: setup-2.12.2-6.el8.noarch”, “Removed: sg3_utils-1.44-5.el8.x86_64”, “Removed: sg3_utils-libs-1.44-5.el8.x86_64”, “Removed: shadow-utils-2:4.6-14.el8.x86_64”, “Removed: sos-4.1-9.el8_5.rocky.3.noarch”, “Removed: sqlite-libs-3.26.0-15.el8.x86_64”, “Removed: sscg-2.3.3-14.el8.x86_64”, “Removed: sssd-client-2.5.2-2.el8_5.4.x86_64”, “Removed: sssd-common-2.5.2-2.el8_5.4.x86_64”, “Removed: sssd-kcm-2.5.2-2.el8_5.4.x86_64”, “Removed: sssd-nfs-idmap-2.5.2-2.el8_5.4.x86_64”, “Removed: sudo-1.8.29-7.el8_4.1.x86_64”, “Removed: systemd-239-51.el8_5.3.x86_64”, “Removed: systemd-libs-239-51.el8_5.3.x86_64”, “Removed: systemd-pam-239-51.el8_5.3.x86_64”, “Removed: systemd-udev-239-51.el8_5.3.x86_64”, “Removed: tar-2:1.30-5.el8.x86_64”, “Removed: teamd-1.31-2.el8.x86_64”, “Removed: tuned-2.16.0-1.el8.noarch”, “Removed: tzdata-2021e-1.el8.noarch”, “Removed: unbound-libs-1.7.3-17.el8.x86_64”, “Removed: util-linux-2.32.1-28.el8.x86_64”, “Removed: vim-minimal-2:8.0.1763-16.el8_5.4.x86_64”, “Removed: virt-what-1.18-12.el8.x86_64”, “Removed: which-2.21-16.el8.x86_64”, “Removed: xfsprogs-5.0.0-9.el8.x86_64”, “Removed: xz-5.2.4-3.el8.1.x86_64”, “Removed: xz-libs-5.2.4-3.el8.1.x86_64”, “Removed: yum-4.7.0-4.el8.noarch”, “Removed: yum-utils-4.0.21-4.el8_5.noarch”, “Removed: zlib-1.2.11-17.el8.x86_64”], “rc”: 0, “invocation”: {“module_args”: {“name”: [“PackageKit”, “PackageKit-glib”, “authselect-compat”, “cairo”, “cairo-gobject”, “cloud-init”, “cloud-init”, “cloud-utils-growpart”, “libestr”, “oddjob”, “oddjob-mkhomedir”, “pixman”, “python3-unbound”, “qemu-guest-agent”, “rsyslog”, “setroubleshoot-server”, “sscg”, “unbound-libs”, “NetworkManager”, “NetworkManager”, “NetworkManager”, “NetworkManager-libnm”, “NetworkManager-libnm”, “NetworkManager-libnm”, “NetworkManager-team”, “NetworkManager-team”, “NetworkManager-team”, “NetworkManager-tui”, “NetworkManager-tui”, “NetworkManager-tui”, “audit”, “audit-libs”, “authselect”, “authselect-libs”, “bash”, “bind-export-libs”, “bind-export-libs”, “c-ares”, “c-ares”, “ca-certificates”, “chrony”, “cockpit-bridge”, “cockpit-system”, “cockpit-ws”, “coreutils”, “coreutils-common”, “cpio”, “cronie”, “cronie-anacron”, “crypto-policies”, “crypto-policies-scripts”, “cryptsetup-libs”, “curl”, “curl”, “dbus”, “dbus”, “dbus-common”, “dbus-common”, “dbus-daemon”, “dbus-daemon”, “dbus-libs”, “dbus-libs”, “dbus-tools”, “dbus-tools”, “dhcp-client”, “dhcp-common”, “dhcp-libs”, “dmidecode”, “dnf”, “dnf-data”, “dnf-plugins-core”, “dracut”, “dracut-config-generic”, “dracut-config-rescue”, “dracut-network”, “dracut-squash”, “e2fsprogs”, “e2fsprogs-libs”, “elfutils-debuginfod-client”, “elfutils-default-yama-scope”, “elfutils-libelf”, “elfutils-libs”, “ethtool”, “expat”, “file”, “file-libs”, “freetype”, “fuse-libs”, “gawk”, “gdbm”, “gdbm-libs”, “gdisk”, “glib2”, “glibc”, “glibc-all-langpacks”, “glibc-common”, “gnupg2”, “gnupg2-smime”, “gnutls”, “gpgme”, “grub2-common”, “grub2-common”, “grub2-pc”, “grub2-pc”, “grub2-pc-modules”, “grub2-pc-modules”, “grub2-tools”, “grub2-tools”, “grub2-tools-extra”, “grub2-tools-extra”, “grub2-tools-minimal”, “grub2-tools-minimal”, “grubby”, “gssproxy”, “gzip”, “hwdata”, “info”, “initscripts”, “iproute”, “iproute”, “iputils”, “irqbalance”, “jansson”, “json-c”, “kernel”, “kernel”, “kernel”, “kernel”, “kernel-core”, “kernel-core”, “kernel-core”, “kernel-core”, “kernel-modules”, “kernel-modules”, “kernel-modules”, “kernel-modules”, “kernel-tools”, “kernel-tools”, “kernel-tools”, “kernel-tools”, “kernel-tools-libs”, “kernel-tools-libs”, “kernel-tools-libs”, “kernel-tools-libs”, “kexec-tools”, “kexec-tools”, “kmod”, “kmod-libs”, “kpartx”, “krb5-libs”, “krb5-libs”, “libarchive”, “libbasicobjects”, “libblkid”, “libbpf”, “libcap”, “libcap”, “libcollection”, “libcom_err”, “libcomps”, “libcurl”, “libcurl”, “libdhash”, “libdnf”, “libfdisk”, “libffi”, “libgcc”, “libgcrypt”, “libgomp”, “libini_config”, “libksba”, “libldb”, “libmount”, “libnfsidmap”, “libnl3”, “libnl3-cli”, “libpath_utils”, “libpwquality”, “libref_array”, “librepo”, “libreport-filesystem”, “libseccomp”, “libsecret”, “libselinux”, “libselinux-utils”, “libsemanage”, “libsmartcols”, “libsolv”, “libsoup”, “libss”, “libssh”, “libssh”, “libssh-config”, “libssh-config”, “libsss_autofs”, “libsss_autofs”, “libsss_certmap”, “libsss_certmap”, “libsss_idmap”, “libsss_idmap”, “libsss_nss_idmap”, “libsss_nss_idmap”, “libsss_sudo”, “libsss_sudo”, “libstdc++”, “libsysfs”, “libtalloc”, “libtasn1”, “libtdb”, “libteam”, “libtevent”, “libtirpc”, “libuser”, “libuuid”, “libverto”, “libverto-libevent”, “libxml2”, “libxml2”, “linux-firmware”, “lmdb-libs”, “logrotate”, “memstrack”, “microcode_ctl”, “microcode_ctl”, “nfs-utils”, “openssh”, “openssh”, “openssh-clients”, “openssh-clients”, “openssh-server”, “openssh-server”, “openssl”, “openssl-libs”, “openssl-pkcs11”, “pam”, “passwd”, “pciutils-libs”, “pcre2”, “platform-python”, “platform-python”, “platform-python-pip”, “platform-python-setuptools”, “policycoreutils”, “policycoreutils-python-utils”, “polkit”, “polkit-libs”, “procps-ng”, “python3-audit”, “python3-dnf”, “python3-dnf-plugins-core”, “python3-gpg”, “python3-hawkey”, “python3-libcomps”, “python3-libdnf”, “python3-libs”, “python3-libs”, “python3-libselinux”, “python3-libsemanage”, “python3-libxml2”, “python3-libxml2”, “python3-linux-procfs”, “python3-perf”, “python3-perf”, “python3-perf”, “python3-perf”, “python3-pip-wheel”, “python3-policycoreutils”, “python3-rpm”, “python3-setools”, “python3-setuptools-wheel”, “python3-slip”, “python3-slip-dbus”, “python3-syspurpose”, “rng-tools”, “rocky-gpg-keys”, “rocky-gpg-keys”, “rocky-logos”, “rocky-release”, “rocky-release”, “rocky-repos”, “rocky-repos”, “rpcbind”, “rpm”, “rpm-build-libs”, “rpm-libs”, “rpm-plugin-selinux”, “rpm-plugin-systemd-inhibit”, “rsync”, “sed”, “selinux-policy”, “selinux-policy”, “selinux-policy-targeted”, “selinux-policy-targeted”, “setup”, “sg3_utils”, “sg3_utils-libs”, “shadow-utils”, “sos”, “sos”, “sos”, “sos”, “sqlite-libs”, “sqlite-libs”, “sssd-client”, “sssd-client”, “sssd-common”, “sssd-common”, “sssd-kcm”, “sssd-kcm”, “sssd-nfs-idmap”, “sssd-nfs-idmap”, “sudo”, “systemd”, “systemd”, “systemd”, “systemd-libs”, “systemd-libs”, “systemd-libs”, “systemd-pam”, “systemd-pam”, “systemd-pam”, “systemd-udev”, “systemd-udev”, “systemd-udev”, “tar”, “teamd”, “tuned”, “tzdata”, “util-linux”, “vim-minimal”, “virt-what”, “which”, “xfsprogs”, “xfsprogs”, “xz”, “xz-libs”, “yum”, “yum-utils”, “zlib”], “state”: “latest”, “allow_downgrade”: false, “autoremove”: false, “bugfix”: false, “cacheonly”: false, “disable_gpg_check”: false, “disable_plugin”: , “disablerepo”: , “download_only”: false, “enable_plugin”: , “enablerepo”: , “exclude”: , “installroot”: “/”, “install_repoquery”: true, “install_weak_deps”: true, “security”: false, “skip_broken”: false, “update_cache”: false, “update_only”: false, “validate_certs”: true, “lock_timeout”: 30, “allowerasing”: false, “nobest”: false, “conf_file”: null, “disable_excludes”: null, “download_dir”: null, “list”: null, “releasever”: null}}}\r\n’, b’‘)
<10.10.0.101> ESTABLISH SSH CONNECTION FOR USER: node-user
<10.10.0.101> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ‘User=“node-user”’ -o ConnectTimeout=10 -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/node-user -o ‘ProxyCommand=ssh -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/bastion-user -W %h:%p -p22 bastion@W.X.Y.Z’ -o ‘ControlPath=“/home/nicolas/test-upgrade-os/config/ansible/cp/35923d7a2f”’ 10.10.0.101 ‘/bin/sh -c ‘"’“‘rm -f -r /home/node-user/.ansible/tmp/ansible-tmp-1691623057.6383471-3972386-257445419832497/ > /dev/null 2>&1 && sleep 0’”’"’’
<10.10.0.101> (0, b’‘, b’')

I diffed the two outputs (failing and working), only to find that the only differences are the following:

  • ansible control paths (expected)
  • ansible temporary dirs (expected)
  • become-success strings (expected)
  • failing run have an “Attempting python interpreter discovery” (line 9 to 13) which doesn’t happen with the working run

Focus on relevant lines:
Attempting python interpreter discovery
<10.10.0.101> ESTABLISH SSH CONNECTION FOR USER: node-user
<10.10.0.101> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ‘User=“node-user”’ -o ConnectTimeout=10 -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/node-user -o ‘ProxyCommand=ssh -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/bastion-user -W %h:%p -p22 bastion@W.X.Y.Z’ -o ‘ControlPath=“/home/nicolas/test-upgrade-os/config/ansible/cp/09896940d7”’ 10.10.0.101 ‘/bin/sh -c ‘"’“'echo PLATFORM; uname; echo FOUND; command -v '”’“'”‘"’“'”‘"’“‘python3.10’”‘"’“'”‘"’“'”‘"’; command -v ‘"’“'”‘"’“'”‘"’“‘python3.9’”‘"’“'”‘"’“'”‘"’; command -v ‘"’“'”‘"’“'”‘"’“‘python3.8’”‘"’“'”‘"’“'”‘"’; command -v ‘"’“'”‘"’“'”‘"’“‘python3.7’”‘"’“'”‘"’“'”‘"’; command -v ‘"’“'”‘"’“'”‘"’“‘python3.6’”‘"’“'”‘"’“'”‘"’; command -v ‘"’“'”‘"’“'”‘"’“‘python3.5’”‘"’“'”‘"’“'”‘"’; command -v ‘"’“'”‘"’“'”‘"’“‘/usr/bin/python3’”‘"’“'”‘"’“'”‘"’; command -v ‘"’“'”‘"’“'”‘"’“‘/usr/libexec/platform-python’”‘"’“'”‘"’“'”‘"’; command -v ‘"’“'”‘"’“'”‘"’“‘python2.7’”‘"’“'”‘"’“'”‘"’; command -v ‘"’“'”‘"’“'”‘"’“‘python2.6’”‘"’“'”‘"’“'”‘"’; command -v ‘"’“'”‘"’“'”‘"’“‘/usr/bin/python’”‘"’“'”‘"’“'”‘"’; command -v ‘"’“'”‘"’“'”‘"’“‘python’”‘"’“'”‘"’“'”‘"’; echo ENDFOUND && sleep 0’“'”‘’
<10.10.0.101> (0, b’PLATFORM\nLinux\nFOUND\n/usr/libexec/platform-python\nENDFOUND\n’, b’‘)
<10.10.0.101> ESTABLISH SSH CONNECTION FOR USER: node-user
<10.10.0.101> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ‘User=“node-user”’ -o ConnectTimeout=10 -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/node-user -o ‘ProxyCommand=ssh -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/bastion-user -W %h:%p -p22 bastion@W.X.Y.Z’ -o ‘ControlPath=“/home/nicolas/test-upgrade-os/config/ansible/cp/09896940d7”’ 10.10.0.101 ‘/bin/sh -c ‘"’"’/usr/libexec/platform-python && sleep 0’“'”‘’
<10.10.0.101> (0, b’{“platform_dist_result”: [“centos”, “8.5”, “Green Obsidian”], “osrelease_content”: “NAME=\“Rocky Linux\”\nVERSION=\“8.5 (Green Obsidian)\”\nID=\“rocky\”\nID_LIKE=\“rhel centos fedora\”\nVERSION_ID=\“8.5\”\nPLATFORM_ID=\“platform:el8\”\nPRETTY_NAME=\“Rocky Linux 8.5 (Green Obsidian)\”\nANSI_COLOR=\“0;32\”\nCPE_NAME=\“cpe:/o:rocky:rocky:8:GA\”\nHOME_URL=\“https://rockylinux.org/\\"\\nBUG_REPORT_URL=\\“https://bugs.rockylinux.org/\\”\\nROCKY_SUPPORT_PRODUCT=\\"Rocky Linux\”\nROCKY_SUPPORT_PRODUCT_VERSION=\“8\”\n”}\n’, b’')

9

I was wondering if the jumphost had anything to do with this issue, so I went ahead and assigned a public IP to one of my hosts.
I then updated my inventory to remove the ProxyCommand option and set the public IP of this host with ansible_host.
Lastly, I ran the following “raw” ansible command: ansible all -u node-user -b --become-user=root -i exec/inventory -m yum -a ‘name=* state=latest’ -vvvv --limit=worker1

The command failed in the same way as with the jumphost. This is the verbose output:
ansible [core 2.12.5]
config file = /home/nicolas/test-upgrade-os/ansible.cfg
configured module search path = [‘/home/nicolas/.ansible/plugins/modules’, ‘/usr/share/ansible/plugins/modules’]
ansible python module location = /home/nicolas/test-upgrade-os/config/venv/lib64/python3.8/site-packages/ansible
ansible collection location = /home/nicolas/.ansible/collections:/usr/share/ansible/collections
executable location = ./config/venv/bin/ansible
python version = 3.8.16 (default, Jun 25 2023, 05:53:51) [GCC 8.5.0 20210514 (Red Hat 8.5.0-18)]
jinja version = 3.1.2
libyaml = True
Using /home/nicolas/test-upgrade-os/ansible.cfg as config file
setting up inventory plugins
host_list declined parsing /home/nicolas/test-upgrade-os/exec/inventory as it did not pass its verify_file() method
script declined parsing /home/nicolas/test-upgrade-os/exec/inventory as it did not pass its verify_file() method
auto declined parsing /home/nicolas/test-upgrade-os/exec/inventory as it did not pass its verify_file() method
Parsed /home/nicolas/test-upgrade-os/exec/inventory inventory source with ini plugin
Loading callback plugin minimal of type stdout, v2.0 from /home/nicolas/test-upgrade-os/config/venv/lib64/python3.8/site-packages/ansible/plugins/callback/minimal.py
Skipping callback ‘default’, as we already have a stdout callback.
Skipping callback ‘minimal’, as we already have a stdout callback.
Skipping callback ‘oneline’, as we already have a stdout callback.
META: ran handlers
<W.X.Y.Z> ESTABLISH SSH CONNECTION FOR USER: node-user
<W.X.Y.Z> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ‘User=“node-user”’ -o ConnectTimeout=10 -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/node-user -o ‘ControlPath=“/home/nicolas/test-upgrade-os/config/ansible/cp/37d3fe42d9”’ W.X.Y.Z ‘/bin/sh -c ‘"’“‘echo ~node-user && sleep 0’”’“‘’
<W.X.Y.Z> (0, b’/home/node-user\n’, b’')
<W.X.Y.Z> ESTABLISH SSH CONNECTION FOR USER: node-user
<W.X.Y.Z> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ‘User=“node-user”’ -o ConnectTimeout=10 -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/node-user -o 'ControlPath=”/home/nicolas/test-upgrade-os/config/ansible/cp/37d3fe42d9"’ W.X.Y.Z ‘/bin/sh -c ‘"’"’( umask 77 && mkdir -p “echo /home/node-user/.ansible/tmp”&& mkdir “echo /home/node-user/.ansible/tmp/ansible-tmp-1691672333.3734102-8205-207838870885533” && echo ansible-tmp-1691672333.3734102-8205-207838870885533=“echo /home/node-user/.ansible/tmp/ansible-tmp-1691672333.3734102-8205-207838870885533” ) && sleep 0’“'”‘’
<W.X.Y.Z> (0, b’ansible-tmp-1691672333.3734102-8205-207838870885533=/home/node-user/.ansible/tmp/ansible-tmp-1691672333.3734102-8205-207838870885533\n’, b’‘)
Attempting python interpreter discovery
<W.X.Y.Z> ESTABLISH SSH CONNECTION FOR USER: node-user
<W.X.Y.Z> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ‘User=“node-user”’ -o ConnectTimeout=10 -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/node-user -o ‘ControlPath=“/home/nicolas/test-upgrade-os/config/ansible/cp/37d3fe42d9”’ W.X.Y.Z ‘/bin/sh -c ‘"’“'echo PLATFORM; uname; echo FOUND; command -v '”’"’“'”‘"’“'”‘“‘python3.10’”’“'”‘"’“'”‘"’“'; command -v '”‘"’“'”‘"’“'”‘“‘python3.9’”’“'”‘"’“'”‘"’“'; command -v '”‘"’“'”‘"’“'”‘“‘python3.8’”’“'”‘"’“'”‘"’“'; command -v '”‘"’“'”‘"’“'”‘“‘python3.7’”’“'”‘"’“'”‘"’“'; command -v '”‘"’“'”‘"’“'”‘“‘python3.6’”’“'”‘"’“'”‘"’“'; command -v '”‘"’“'”‘"’“'”‘“‘python3.5’”’“'”‘"’“'”‘"’“'; command -v '”‘"’“'”‘"’“'”‘"’/usr/bin/python3’“'”‘"’“'”‘"’“'”‘; command -v ‘"’"’“'”‘"’“'”‘"’/usr/libexec/platform-python’“'”‘"’“'”‘"’“'”‘; command -v ‘"’"’“'”‘"’“'”‘“‘python2.7’”’“'”‘"’“'”‘"’“'; command -v '”‘"’“'”‘"’“'”‘“‘python2.6’”’“'”‘"’“'”‘"’“'; command -v '”‘"’“'”‘"’“'”‘"’/usr/bin/python’“'”‘"’“'”‘"’“'”‘; command -v ‘"’"’“'”‘"’“'”‘“‘python’”’“'”‘"’“'”‘"’“‘; echo ENDFOUND && sleep 0’”‘"’’
<W.X.Y.Z> (0, b’PLATFORM\nLinux\nFOUND\n/usr/libexec/platform-python\nENDFOUND\n’, b’‘)
<W.X.Y.Z> ESTABLISH SSH CONNECTION FOR USER: node-user
<W.X.Y.Z> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ‘User=“node-user”’ -o ConnectTimeout=10 -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/node-user -o ‘ControlPath=“/home/nicolas/test-upgrade-os/config/ansible/cp/37d3fe42d9”’ W.X.Y.Z ‘/bin/sh -c ‘"’"’/usr/libexec/platform-python && sleep 0’“'”‘’
<W.X.Y.Z> (0, b’{“platform_dist_result”: [“centos”, “8.5”, “Green Obsidian”], “osrelease_content”: “NAME=\“Rocky Linux\”\nVERSION=\“8.5 (Green Obsidian)\”\nID=\“rocky\”\nID_LIKE=\“rhel centos fedora\”\nVERSION_ID=\“8.5\”\nPLATFORM_ID=\“platform:el8\”\nPRETTY_NAME=\“Rocky Linux 8.5 (Green Obsidian)\”\nANSI_COLOR=\“0;32\”\nCPE_NAME=\“cpe:/o:rocky:rocky:8:GA\”\nHOME_URL=\“https://rockylinux.org/\\"\\nBUG_REPORT_URL=\\“https://bugs.rockylinux.org/\\”\\nROCKY_SUPPORT_PRODUCT=\\"Rocky Linux\”\nROCKY_SUPPORT_PRODUCT_VERSION=\“8\”\n”}\n’, b’‘)
Using module file /home/nicolas/test-upgrade-os/config/venv/lib64/python3.8/site-packages/ansible/modules/setup.py
<W.X.Y.Z> PUT /home/nicolas/test-upgrade-os/config/ansible/tmp/ansible-local-81995b5ndxdt/tmpn6x30tgz TO /home/node-user/.ansible/tmp/ansible-tmp-1691672333.3734102-8205-207838870885533/AnsiballZ_setup.py
<W.X.Y.Z> SSH: EXEC sftp -b - -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ‘User=“node-user”’ -o ConnectTimeout=10 -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/node-user -o ‘ControlPath=“/home/nicolas/test-upgrade-os/config/ansible/cp/37d3fe42d9”’ ‘[W.X.Y.Z]’
<W.X.Y.Z> (0, b’sftp> put /home/nicolas/test-upgrade-os/config/ansible/tmp/ansible-local-81995b5ndxdt/tmpn6x30tgz /home/node-user/.ansible/tmp/ansible-tmp-1691672333.3734102-8205-207838870885533/AnsiballZ_setup.py\n’, b’‘)
<W.X.Y.Z> ESTABLISH SSH CONNECTION FOR USER: node-user
<W.X.Y.Z> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ‘User=“node-user”’ -o ConnectTimeout=10 -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/node-user -o ‘ControlPath=“/home/nicolas/test-upgrade-os/config/ansible/cp/37d3fe42d9”’ W.X.Y.Z ‘/bin/sh -c ‘"’“‘chmod u+x /home/node-user/.ansible/tmp/ansible-tmp-1691672333.3734102-8205-207838870885533/ /home/node-user/.ansible/tmp/ansible-tmp-1691672333.3734102-8205-207838870885533/AnsiballZ_setup.py && sleep 0’”’"’’
<W.X.Y.Z> (0, b’‘, b’‘)
<W.X.Y.Z> ESTABLISH SSH CONNECTION FOR USER: node-user
<W.X.Y.Z> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ‘User=“node-user”’ -o ConnectTimeout=10 -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/node-user -o ‘ControlPath=“/home/nicolas/test-upgrade-os/config/ansible/cp/37d3fe42d9”’ W.X.Y.Z ‘/bin/sh -c ‘"’"’( umask 77 && mkdir -p “echo /home/node-user/.ansible/tmp”&& mkdir “echo /home/node-user/.ansible/tmp/ansible-tmp-1691672333.3734102-8205-207838870885533” && echo ansible-tmp-1691672333.3734102-8205-207838870885533=“echo /home/node-user/.ansible/tmp/ansible-tmp-1691672333.3734102-8205-207838870885533” ) && sleep 0’“'”‘’
<W.X.Y.Z> (0, b’ansible-tmp-1691672333.3734102-8205-207838870885533=/home/node-user/.ansible/tmp/ansible-tmp-1691672333.3734102-8205-207838870885533\n’, b’‘)
Attempting python interpreter discovery
<W.X.Y.Z> ESTABLISH SSH CONNECTION FOR USER: node-user
<W.X.Y.Z> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ‘User=“node-user”’ -o ConnectTimeout=10 -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/node-user -o ‘ControlPath=“/home/nicolas/test-upgrade-os/config/ansible/cp/37d3fe42d9”’ W.X.Y.Z ‘/bin/sh -c ‘"’“'echo PLATFORM; uname; echo FOUND; command -v '”’"’“'”‘"’“'”‘“‘python3.10’”’“'”‘"’“'”‘"’“'; command -v '”‘"’“'”‘"’“'”‘“‘python3.9’”’“'”‘"’“'”‘"’“'; command -v '”‘"’“'”‘"’“'”‘“‘python3.8’”’“'”‘"’“'”‘"’“'; command -v '”‘"’“'”‘"’“'”‘“‘python3.7’”’“'”‘"’“'”‘"’“'; command -v '”‘"’“'”‘"’“'”‘“‘python3.6’”’“'”‘"’“'”‘"’“'; command -v '”‘"’“'”‘"’“'”‘“‘python3.5’”’“'”‘"’“'”‘"’“'; command -v '”‘"’“'”‘"’“'”‘"’/usr/bin/python3’“'”‘"’“'”‘"’“'”‘; command -v ‘"’"’“'”‘"’“'”‘"’/usr/libexec/platform-python’“'”‘"’“'”‘"’“'”‘; command -v ‘"’"’“'”‘"’“'”‘“‘python2.7’”’“'”‘"’“'”‘"’“'; command -v '”‘"’“'”‘"’“'”‘“‘python2.6’”’“'”‘"’“'”‘"’“'; command -v '”‘"’“'”‘"’“'”‘"’/usr/bin/python’“'”‘"’“'”‘"’“'”‘; command -v ‘"’"’“'”‘"’“'”‘“‘python’”’“'”‘"’“'”‘"’“‘; echo ENDFOUND && sleep 0’”‘"’’
<W.X.Y.Z> (0, b’PLATFORM\nLinux\nFOUND\n/usr/libexec/platform-python\nENDFOUND\n’, b’‘)
<W.X.Y.Z> ESTABLISH SSH CONNECTION FOR USER: node-user
<W.X.Y.Z> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ‘User=“node-user”’ -o ConnectTimeout=10 -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/node-user -o ‘ControlPath=“/home/nicolas/test-upgrade-os/config/ansible/cp/37d3fe42d9”’ W.X.Y.Z ‘/bin/sh -c ‘"’"’/usr/libexec/platform-python && sleep 0’“'”‘’
<W.X.Y.Z> (0, b’{“platform_dist_result”: [“centos”, “8.5”, “Green Obsidian”], “osrelease_content”: “NAME=\“Rocky Linux\”\nVERSION=\“8.5 (Green Obsidian)\”\nID=\“rocky\”\nID_LIKE=\“rhel centos fedora\”\nVERSION_ID=\“8.5\”\nPLATFORM_ID=\“platform:el8\”\nPRETTY_NAME=\“Rocky Linux 8.5 (Green Obsidian)\”\nANSI_COLOR=\“0;32\”\nCPE_NAME=\“cpe:/o:rocky:rocky:8:GA\”\nHOME_URL=\“https://rockylinux.org/\\"\\nBUG_REPORT_URL=\\“https://bugs.rockylinux.org/\\”\\nROCKY_SUPPORT_PRODUCT=\\"Rocky Linux\”\nROCKY_SUPPORT_PRODUCT_VERSION=\“8\”\n”}\n’, b’‘)
Using module file /home/nicolas/test-upgrade-os/config/venv/lib64/python3.8/site-packages/ansible/modules/setup.py
<W.X.Y.Z> PUT /home/nicolas/test-upgrade-os/config/ansible/tmp/ansible-local-81995b5ndxdt/tmpn6x30tgz TO /home/node-user/.ansible/tmp/ansible-tmp-1691672333.3734102-8205-207838870885533/AnsiballZ_setup.py
<W.X.Y.Z> SSH: EXEC sftp -b - -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ‘User=“node-user”’ -o ConnectTimeout=10 -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/node-user -o ‘ControlPath=“/home/nicolas/test-upgrade-os/config/ansible/cp/37d3fe42d9”’ ‘[W.X.Y.Z]’
<W.X.Y.Z> (0, b’sftp> put /home/nicolas/test-upgrade-os/config/ansible/tmp/ansible-local-81995b5ndxdt/tmpn6x30tgz /home/node-user/.ansible/tmp/ansible-tmp-1691672333.3734102-8205-207838870885533/AnsiballZ_setup.py\n’, b’‘)
<W.X.Y.Z> ESTABLISH SSH CONNECTION FOR USER: node-user
<W.X.Y.Z> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ‘User=“node-user”’ -o ConnectTimeout=10 -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/node-user -o ‘ControlPath=“/home/nicolas/test-upgrade-os/config/ansible/cp/37d3fe42d9”’ W.X.Y.Z ‘/bin/sh -c ‘"’“‘chmod u+x /home/node-user/.ansible/tmp/ansible-tmp-1691672333.3734102-8205-207838870885533/ /home/node-user/.ansible/tmp/ansible-tmp-1691672333.3734102-8205-207838870885533/AnsiballZ_setup.py && sleep 0’”’"’’
<W.X.Y.Z> (0, b’‘, b’‘)
<W.X.Y.Z> ESTABLISH SSH CONNECTION FOR USER: node-user
<W.X.Y.Z> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ‘User=“node-user”’ -o ConnectTimeout=10 -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/node-user -o ‘ControlPath=“/home/nicolas/test-upgrade-os/config/ansible/cp/37d3fe42d9”’ -tt W.X.Y.Z ‘/bin/sh -c ‘"’“'sudo -H -S -n -u root /bin/sh -c '”’"’“'”‘"’“'”‘“‘echo BECOME-SUCCESS-rcdtreshgiukhhtbaenoexpcbqriojgw ; /usr/libexec/platform-python /home/node-user/.ansible/tmp/ansible-tmp-1691672333.3734102-8205-207838870885533/AnsiballZ_setup.py’”’“'”‘"’“'”‘"’“’ && sleep 0’”‘"’’
Escalation succeeded
<W.X.Y.Z> (0, b’\r\n{“ansible_facts”: {“ansible_pkg_mgr”: “dnf”}, “invocation”: {“module_args”: {“filter”: [“ansible_pkg_mgr”], “gather_subset”: [“!all”], “gather_timeout”: 10, “fact_path”: “/etc/ansible/facts.d”}}}\r\n’, b’‘)
Running ansible.legacy.dnf as the backend for the yum action plugin
Using module file /home/nicolas/test-upgrade-os/config/venv/lib64/python3.8/site-packages/ansible/modules/dnf.py
<W.X.Y.Z> PUT /home/nicolas/test-upgrade-os/config/ansible/tmp/ansible-local-81995b5ndxdt/tmphjjuz1pb TO /home/node-user/.ansible/tmp/ansible-tmp-1691672333.3734102-8205-207838870885533/AnsiballZ_dnf.py
<W.X.Y.Z> SSH: EXEC sftp -b - -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ‘User=“node-user”’ -o ConnectTimeout=10 -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/node-user -o ‘ControlPath=“/home/nicolas/test-upgrade-os/config/ansible/cp/37d3fe42d9”’ ‘[W.X.Y.Z]’
<W.X.Y.Z> (0, b’sftp> put /home/nicolas/test-upgrade-os/config/ansible/tmp/ansible-local-81995b5ndxdt/tmphjjuz1pb /home/node-user/.ansible/tmp/ansible-tmp-1691672333.3734102-8205-207838870885533/AnsiballZ_dnf.py\n’, b’‘)
<W.X.Y.Z> ESTABLISH SSH CONNECTION FOR USER: node-user
<W.X.Y.Z> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ‘User=“node-user”’ -o ConnectTimeout=10 -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/node-user -o ‘ControlPath=“/home/nicolas/test-upgrade-os/config/ansible/cp/37d3fe42d9”’ W.X.Y.Z ‘/bin/sh -c ‘"’“‘chmod u+x /home/node-user/.ansible/tmp/ansible-tmp-1691672333.3734102-8205-207838870885533/ /home/node-user/.ansible/tmp/ansible-tmp-1691672333.3734102-8205-207838870885533/AnsiballZ_dnf.py && sleep 0’”’"’’
<W.X.Y.Z> (0, b’‘, b’‘)
<W.X.Y.Z> ESTABLISH SSH CONNECTION FOR USER: node-user
<W.X.Y.Z> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ‘User=“node-user”’ -o ConnectTimeout=10 -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/node-user -o ‘ControlPath=“/home/nicolas/test-upgrade-os/config/ansible/cp/37d3fe42d9”’ -tt W.X.Y.Z ‘/bin/sh -c ‘"’“'sudo -H -S -n -u root /bin/sh -c '”’"’“'”‘"’“'”‘“‘echo BECOME-SUCCESS-slnsgexbjhxxsqxeuoxtmbwjmrqrpsmw ; /usr/libexec/platform-python /home/node-user/.ansible/tmp/ansible-tmp-1691672333.3734102-8205-207838870885533/AnsiballZ_dnf.py’”’“'”‘"’“'”‘"’“’ && sleep 0’”‘"’’
Escalation succeeded
<W.X.Y.Z> (255, b’‘, b’‘)
<W.X.Y.Z> ESTABLISH SSH CONNECTION FOR USER: node-user
<W.X.Y.Z> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ‘User=“node-user”’ -o ConnectTimeout=10 -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/node-user -o ‘ControlPath=“/home/nicolas/test-upgrade-os/config/ansible/cp/37d3fe42d9”’ W.X.Y.Z ‘/bin/sh -c ‘"’“‘rm -f -r /home/node-user/.ansible/tmp/ansible-tmp-1691672333.3734102-8205-207838870885533/ > /dev/null 2>&1 && sleep 0’”’"’’
<W.X.Y.Z> (0, b’‘, b’')
worker1 | UNREACHABLE! => {
“changed”: false,
“msg”: "Failed to connect to the host via ssh: ",
“unreachable”: true
}

As usual, the “yum update” did succeed on the host:
ID | Command line | Date and time | Action(s) | Altered

10

So you are getting errors on the history in transaction 8, might want to run “yum history info 8” and see what they are. IF the task is actually completing despite the role failing, Passed on that message, looks like the issue is in happen when ansible is reconnecting.

This block here looks almost like you are shooting ansible in the head. ‘rm -f -r /home/node-user/.ansible/tmp/ansible-tmp-1691672333.3734102-8205-207838870885533/ > /dev/null 2>&1 && sleep 0’ as it is deleting the location ansible is looking for out from under itself.

<W.X.Y.Z> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ‘User=“node-user”’ -o ConnectTimeout=10 -q -o UserKnownHostsFile=ssh/known_hosts -i ssh/node-user -o ‘ControlPath=“/home/nicolas/test-upgrade-os/config/ansible/cp/37d3fe42d9”’ W.X.Y.Z ‘/bin/sh -c ‘"’“‘rm -f -r /home/node-user/.ansible/tmp/ansible-tmp-1691672333.3734102-8205-207838870885533/ > /dev/null 2>&1 && sleep 0’”’"‘’
<W.X.Y.Z> (0, b’‘, b’')
worker1 | UNREACHABLE! => {
“changed”: false,
“msg”: "Failed to connect to the host via ssh: ",

“unreachable”: true

11

I’m learning things along the way, thank you :slight_smile: Here is the output of “yum history info 7” (yeah, I did recreate my VMs so that’s why it’s a different ID):

Transaction ID : 7
Begin time : Thu 10 Aug 2023 02:36:15 PM UTC
Begin rpmdb : 430:cf802f1003554ad70c79c2569d6fae2fb17ca591
End time : Thu 10 Aug 2023 02:46:12 PM UTC (9 minutes)
End rpmdb : 439:e567b314bae50c45cb6d63fbcaea7a4033f5cf42
User : node-user
Return-Code : Success
Releasever : 8
Command Line :
Comment :
Packages Altered:
Install python3-netifaces-0.10.6-4.el8.x86_64 @appstream
Install NetworkManager-initscripts-updown-1:1.40.16-4.el8_8.noarch @baseos
Install glibc-gconv-extra-2.28-225.el8.x86_64 @baseos
Install grub2-tools-efi-1:2.02-148.el8_8.1.rocky.0.3.x86_64 @baseos
Install kernel-4.18.0-477.21.1.el8_8.x86_64 @baseos
Install kernel-core-4.18.0-477.21.1.el8_8.x86_64 @baseos
Install kernel-modules-4.18.0-477.21.1.el8_8.x86_64 @baseos
Install python3-magic-5.33-24.el8.noarch @baseos
Install python3-setuptools-39.2.0-7.el8.noarch @baseos
Upgrade PackageKit-1.1.12-6.el8.0.2.x86_64 @appstream
Upgraded PackageKit-1.1.12-6.el8.x86_64 @@System
Upgrade PackageKit-glib-1.1.12-6.el8.0.2.x86_64 @appstream
Upgraded PackageKit-glib-1.1.12-6.el8.x86_64 @@System
Upgrade authselect-compat-1.2.6-1.el8.x86_64 @appstream
Upgraded authselect-compat-1.2.2-3.el8.x86_64 @@System
Upgrade cairo-1.15.12-6.el8.x86_64 @appstream
Upgraded cairo-1.15.12-3.el8.x86_64 @@System
Upgrade cairo-gobject-1.15.12-6.el8.x86_64 @appstream
Upgraded cairo-gobject-1.15.12-3.el8.x86_64 @@System
(sparing you the 500 lines here)
Upgrade yum-4.7.0-16.el8_8.noarch @baseos
Upgraded yum-4.7.0-4.el8.noarch @@System
Upgrade yum-utils-4.0.21-19.el8_8.noarch @baseos
Upgraded yum-utils-4.0.21-4.el8_5.noarch @@System
Upgrade zlib-1.2.11-21.el8_7.x86_64 @baseos
Upgraded zlib-1.2.11-17.el8.x86_64 @@System
Scriptlet output:
1 warning: /etc/shadow created as /etc/shadow.rpmnew
2 warning: /etc/systemd/logind.conf created as /etc/systemd/logind.conf.rpmnew
3 warning: /etc/cloud/cloud.cfg created as /etc/cloud/cloud.cfg.rpmnew
4 libsemanage.semanage_direct_install_info: Overriding cockpit module at lower priority 100 with module at priority 200.
5 warning: /etc/ssh/sshd_config created as /etc/ssh/sshd_config.rpmnew

Other than the four warnings, do you think the output of scriplet #4 (libsemanage.semanage_direct_install_info) could be the culprit here ? Also, I didn’t mention it earlier, but SELinux is disabled:

$ sestatus
SELinux status: disabled

Regarding the rm -f -r /home/node-user/.ansible/tmp/ansible-tmp-1691672333.3734102-8205-207838870885533/ > /dev/null 2>&1 && sleep 0, I did laugh when I saw your message, but I didn’t ask for that specifically. I think this is just ansible cleaning its temporary directory before finishing its run.

12

I’m really sorry but I keep posting messages that have to be approved and they are not until several hours… Is there a way for my messages to be auto-approved ?
This makes the process really slow and quite painful…

Anyway, just to let you know I tried running the “shell” module with “yum update -y” as the command and I have the exact same behavior : the process ends in error but the host packages are successfully upgraded.
So in the end this is not really related to neither one of the “dnf”, “yum” or “package” modules, but rather on something not working correctly on my host.

I’ll repost the result of “yum history info”:
Scriptlet output:
1 warning: /etc/shadow created as /etc/shadow.rpmnew
2 warning: /etc/systemd/logind.conf created as /etc/systemd/logind.conf.rpmnew
3 warning: /etc/cloud/cloud.cfg created as /etc/cloud/cloud.cfg.rpmnew
4 libsemanage.semanage_direct_install_info: Overriding cockpit module at lower priority 100 with module at priority 200.
5 warning: /etc/ssh/sshd_config created as /etc/ssh/sshd_config.rpmnew

I didn’t mention it earlier, but SELinux is disabled on my hosts.
I wouldn’t expect the warnings of new “rpmnew” files to be the culprit here, but maybe libsemanage is ? I wouldn’t understand why, as this is related to SELinux, which is disabled.