# unreachable and retries

**URL:** <https://forum.ansible.com/t/unreachable-and-retries/37785>\
**Category:** Ansible Project\
**Created:** [March 20, 2023, 4:57pm UTC](https://forum.ansible.com/t/unreachable-and-retries/37785 "2023-03-20T16:57:55Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Francisco\_Palomares](https://avatars.discourse-cdn.com/v4/letter/f/47e85d/32.png) [@Francisco\_Palomares](https://forum.ansible.com/u/Francisco_Palomares)\
**Post date:** [March 20, 2023, 4:57pm UTC](https://forum.ansible.com/t/unreachable-and-retries/37785/1 "2023-03-20T16:57:55Z")

</div>

Hi Ansible Experts !

I am working in a playbook that executes a long running task (patching), that if the session gets disconnected half-way in the execution can cause serious damage in the target server.

Using async module with retries seems the right approach, I hope.

I am now trying to handle the scenario of reporting that we had an “unreachable” error while the async job is running.

We could have intermittent ldap and network issues. Thanks to async, the patch shell script won’t stop running… but I want to inform the engineers (email, dashboard, etc.) that the playbook have actually stopped due to “unreachable” error.

But apparently ignore\_unreachable is not working as I expected, when used with retry.

Based in the feedback given in this issue [https://github.com/ansible/ansible/issues/78358](https://github.com/ansible/ansible/issues/78358), ignore\_unreachable is not honored with retries.

I am not really looking for “keep retrying” functionality, but at least that I can get a failed task so I can rescue properly afterwards.

This is an example of what I am trying to achieve:

- name: patching  
become: yes  
block:

- name: run patching async  
async: 43200  
poll: 0  
shell: my\_patch.sh  
register: patch\_sleeper

- name: wait for async job to end  
async\_status:  
jid: ‘{{ patch\_sleeper.ansible\_job\_id }}’  
register: job\_result  
until: job\_result.finished  
retries: 720  
delay: 1  
ignore\_unreachable: true

- name: error handling for unreachable in the middle of the run  
fail:  
msg: Detected unreachable host error. Forcing a fail to trigger any rescue.  
when: job\_result.unreachable is defined  
rescue:

- name: send message to Monitoring Dashboard  
my\_method:  
message: there was an error during patching

And this is an extract of the output I get, that clearly shows that the ignore\_unreachable = true is ignored (no pun intended 🙂 )

(ansible 2.12)

TASK [wait for async job to end] \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*  
FAILED - RETRYING: [mynode]: wait for async job to end (720 retries left).  
…  
FAILED - RETRYING: [mynode]: wait for async job to end (705 retries left).  
fatal: [mynode]: UNREACHABLE! =\> changed=false  
msg: 'Failed to connect to the host via ssh: ’  
skip\_reason: Host mynode is unreachable  
unreachable: true

NO MORE HOSTS LEFT \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*

I explored developing my own custom\_async\_status, callbacks or action\_plugins … but none of those seems capable to change the status from “unreachable” to “failed”.

Is there any way to convert status “unreachable” into a “failed” so that can be rescued? Or to somehow make ignore\_unreachable working with retries?

Thanks in advance,

FP

---

<div class="post-metadata">

**Author:** ![shearn](https://avatars.discourse-cdn.com/v4/letter/s/ba8739/32.png) [@shearn](https://forum.ansible.com/u/shearn)\
**Post date:** [March 20, 2023, 7:57pm UTC](https://forum.ansible.com/t/unreachable-and-retries/37785/2 "2023-03-20T19:57:47Z")

</div>

I’ve found that if you have an unreliable connection to a target, you have at least two challenges. One is Ansible and another is the package repo connection instability. Sometimes they both recover or finish and sometimes they don’t.

- create an execution environment ansible control node nearest to the edge system with a more reliable connection to the target. Run your play there.

or

- schedule and run your play on the target system that you are patching. The target becomes the Ansible Control Node. Push the play to the target and then schedule it to run and then inspect the local logs for completion.

Troubleshooting flaky and unreliable connections is most difficult.

There are probably many other ways to manage this.

Regards,

Stan