When running a playbook for updating packages on our RHEL9.x systems, the stdout from ansible-playbook (at any verbosity level) hangs at a ansible.builtin.dnf task.
If I don’t ctrl-c out of the process in the terminal, i got an error like
fatal: [node-hostname-example]: FAILED! =>
changed: false
msg: The ansible.builtin.dnf action failed to execute in the expected time frame (1800)
and was terminated
timedout:
frame: Interrupted at <frame at 0x7f6fba5f6a40, file '/usr/lib64/python3.12/selectors.py',
line 468, code select> called from <frame at 0x7f6fba87b870, file '/os/rhel9/ansible/venv/ansible-core_2.18/lib64/python3.12/site-packages/ansible/executor/task_executor.py',
line 673, code _execute>
period: 1800
NOTE: The task_timeout is set to 1800 seconds in the ansible.cfg
What’s odd is that if I log into machines I was trying to update and check versions of packages that needed updating, they all have pushed to their latest version, indicating that the dnf update task was successful.
But then why did the info never make it back to me and the playbook hang?
I’m experiencing a similar issue on some RHEL9 systems, and I’ve found that it usually has to do with howI think I know where the issue might be. It looks like the DNF update task is actually succeeding on the remote hosts, but the stdout is hanging due to its verbosity. Have you tried setting the ‘async’ and ‘poll’ options on the DNF task? This can help control the output and prevent it from hanging.
Here’s an example of how you can do it:
- name: upgrade packages in BaseOS & Appstream
ansible.builtin.dnf:
...
async: 300 # set a timeout of 5 minutes
poll: 0 # poll every 0 seconds (i.e. don't wait for the task to finish)
This way, the task runs asynchronously and the playbook doesn’t hang while waiting for output. It’s worked reliably in my tests.
Such behavior can be reproduced with packages requiring interaction during updates, like press Y, agree to an EULA, etc. It could become addressed like
There a bit of confusion here. The deprecated async option is for the ansible.builtin.yum_repository module, not the yum or dnf modules.
The async and poll options are not options of the dnf module, but rather of individual tasks. That is, rather than
- name: upgrade packages in BaseOS & Appstream
ansible.builtin.dnf:
...
async: 300 # Broken! Not a "dnf" option. Set a timeout of 5 minutes
poll: 0 # Broken! Not a "dnf" option. Poll every 0 seconds (i.e. don't wait for the task to finish)
which has incorrect indentation, it should be
- name: upgrade packages in BaseOS & Appstream
ansible.builtin.dnf:
...
async: 300 # Fixed! Task option. Set a timeout of 5 minutes
poll: 0 # Fixed! Task option. Poll every 0 seconds (i.e. don't wait for the task to finish)
Knowledge of the relationship between action plugins and, er, “everything else” is not widespread (I’m projecting here) mostly because one can get a lot done in Ansible without it. I think I’ve understood it briefly a few times before, but such understanding evaporates with disuse. I should go document diving again…
it is mostly transparent, every task always executes an action plugin, either one that matches the action specified on the task (or via collection configuration) or the normal action plugin, which is thin wrapper to module that matches action specified on the task.
They are normally used to run things required on the controller before pushing things to the remote (template and copy are good examples). In the case of package/service/dnf/yum they run to ‘figure out’ what to run on the remote, mostly by either using existing facts or gathering themselves if needed.
I get the list below and I do not have enough experience to say that any particular package jumps out at me. When I run the update on the command line, I don’t get any prompt for a EULA or a needed restart. The NetworkManager service needs a restart (which is the next task in the playbook), but no reason for a task to timeout.