# Trying to speed up ansible with serial running, but it hangs after a while constantly

**URL:** <https://forum.ansible.com/t/trying-to-speed-up-ansible-with-serial-running-but-it-hangs-after-a-while-constantly/40370>\
**Category:** Get Help\
**Created:** [February 19, 2025, 4:14pm UTC](https://forum.ansible.com/t/trying-to-speed-up-ansible-with-serial-running-but-it-hangs-after-a-while-constantly/40370 "2025-02-19T16:14:21Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![lilian-granqvist](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/lilian-granqvist/32/4731_2.png) [@lilian-granqvist](https://forum.ansible.com/u/lilian-granqvist)\
**Post date:** [February 19, 2025, 4:14pm UTC](https://forum.ansible.com/t/trying-to-speed-up-ansible-with-serial-running-but-it-hangs-after-a-while-constantly/40370/1 "2025-02-19T16:14:21Z")

</div>

We have some hundred servers I try to run ansible script against, if I’ve understood correctly it by default runs one host at a time, and with serial you can increase this. depending on if I have 100% or 10 as serial it just crashes at some point.

With serial: 10 i had 10 ssh connections constantly until when it hanged, after which i had only 1 and network usage spiked

```auto
---
- name: "Add or remove users"
  hosts: tag_ansible_managed_true
  # serial:
  # - 10
  become: true
  gather_facts: true
  strategy: free # noqa: run-once[play]
  roles:
    - amazon_user_accounts
...

```

 ![image](https://us1.discourse-cdn.com/ansible/original/2X/9/99c5c3efd6bb2741f37ded810515e9262b237772.png)

---

<div class="post-metadata">

**Author:** ![lilian-granqvist](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/lilian-granqvist/32/4731_2.png) [@lilian-granqvist](https://forum.ansible.com/u/lilian-granqvist)\
**Post date:** [February 19, 2025, 4:47pm UTC](https://forum.ansible.com/t/trying-to-speed-up-ansible-with-serial-running-but-it-hangs-after-a-while-constantly/40370/2 "2025-02-19T16:47:43Z")

</div>

Seems it’s not serial, but it’s always at the exact same position and ends in a situation like in the screenshot

(at ctrl+c point ansible had been running for 37m 44s possibility of some timeout at 35min ? )

---

<div class="post-metadata">

**Author:** ![sivel](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/sivel/32/1198_2.png) [@sivel](https://forum.ansible.com/u/sivel)\
**Post date:** [February 19, 2025, 5:32pm UTC](https://forum.ansible.com/t/trying-to-speed-up-ansible-with-serial-running-but-it-hangs-after-a-while-constantly/40370/3 "2025-02-19T17:32:43Z")

</div>

It could be that you are experiencing network issues, and although the SSH connection is dead, the ssh client hasn’t recognized it. You may want to add `-o ServerAliveInterval=60s` or some other value to your [`ssh_extra_args`](https://docs.ansible.com/ansible/latest/collections/ansible/builtin/ssh_connection.html#parameter-ssh_extra_args).

It’s hard to say without more info. You might try using `strace` to see if you can determine what activity is happening in the processes.

---

<div class="post-metadata">

**Author:** ![anon81884396](https://avatars.discourse-cdn.com/v4/letter/a/b782af/32.png) [@anon81884396](https://forum.ansible.com/u/anon81884396)\
**Post date:** [February 19, 2025, 5:50pm UTC](https://forum.ansible.com/t/trying-to-speed-up-ansible-with-serial-running-but-it-hangs-after-a-while-constantly/40370/4 "2025-02-19T17:50:24Z")

</div>

Tough to say without more info but one thing I’ve encountered in the past is a dead NFS or NAS mount on a target machine causing my job to hang during the gather\_facts stage

---

<div class="post-metadata">

**Author:** ![lilian-granqvist](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/lilian-granqvist/32/4731_2.png) [@lilian-granqvist](https://forum.ansible.com/u/lilian-granqvist)\
**Post date:** [February 19, 2025, 5:57pm UTC](https://forum.ansible.com/t/trying-to-speed-up-ansible-with-serial-running-but-it-hangs-after-a-while-constantly/40370/5 "2025-02-19T17:57:59Z")

</div>

Yeah i found some tcpkeepalive and serveraliveinterval options i tried to add  
Currently ssh\_args looks like this:

> [ssh\_connection]  
> ssh\_args=-o ServerAliveInterval=30 -o ServerAliveCountMax=3 -o TCPKeepAlive=yes -o ControlMaster=auto -o ControlPersist=60s -o ControlPath=/tmp/ansible-ssh-%h-%p-%r

It always dies after the first of these two tasks are complete like 50servers in

```auto
TASK [amazon_user_accounts : Configure /etc/sudoers.d/] ***************************************************************************************************

TASK [amazon_user_accounts : Set authorized_key for the user (exclusive)] *********************************************************************************

```

Trying to strace the process, i’ll inform if i find anything more

---

<div class="post-metadata">

**Author:** ![lilian-granqvist](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/lilian-granqvist/32/4731_2.png) [@lilian-granqvist](https://forum.ansible.com/u/lilian-granqvist)\
**Post date:** [February 19, 2025, 6:16pm UTC](https://forum.ansible.com/t/trying-to-speed-up-ansible-with-serial-running-but-it-hangs-after-a-while-constantly/40370/6 "2025-02-19T18:16:47Z")

</div>

not sure what i’m looking for in strace, but when ansible hangs up the process just spams nonstop this line :

```auto
20:08:06 clock_nanosleep(CLOCK_MONOTONIC, TIMER_ABSTIME, {tv_sec=28132, tv_nsec=516224116}, NULL) = 0

```

Depending on batch sizes it always stops after the same servers task is complete and is supposed to start next task

---

<div class="post-metadata">

**Author:** ![sivel](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/sivel/32/1198_2.png) [@sivel](https://forum.ansible.com/u/sivel)\
**Post date:** [February 19, 2025, 7:02pm UTC](https://forum.ansible.com/t/trying-to-speed-up-ansible-with-serial-running-but-it-hangs-after-a-while-constantly/40370/7 "2025-02-19T19:02:33Z")

</div>

You might have to log into the affected server that ssh is still open to, and determine what the module execution is doing.

Ultimately it sounds like ansible is waiting to get a module result back from a server that is never coming. When that happens it’s usually 1 of 2 things:

1. The connection is dead and ssh hasn’t noticed
2. the module on the remote side is hung for some reason

If that result never comes, ansible will appear to just hang.

You can check out this gist for attempting to diagnose why a module execution may be happening on a target host: [How to debug a hanging ansible module · GitHub](https://gist.github.com/sivel/5076af99d310dbda3188722f02cddeb4)

---

<div class="post-metadata">

**Author:** ![lilian-granqvist](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/lilian-granqvist/32/4731_2.png) [@lilian-granqvist](https://forum.ansible.com/u/lilian-granqvist)\
**Post date:** [February 19, 2025, 7:24pm UTC](https://forum.ansible.com/t/trying-to-speed-up-ansible-with-serial-running-but-it-hangs-after-a-while-constantly/40370/8 "2025-02-19T19:24:18Z")

</div>

I found the issue 😮‍💨 , somewhere at somepoint ssh tried to ask me if i trust a new servers ip adress i hadn’t yet connected to…  
(i saw this after i outputted the ansible stuff to a file instead of console)

```auto
Playbook run took 0 days, 0 hours, 11 minutes, 58 seconds with a serial batch size of 10 and 10 forks.

```

I think i’ll try and make a loop before the main task to go through all servers and see that i can connect to them, should be more easily visible then i think 🤔

---

<div class="post-metadata">

**Author:** ![hugonz](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/hugonz/32/503_2.png) [@hugonz](https://forum.ansible.com/u/hugonz)\
**Post date:** [February 20, 2025, 11:16pm UTC](https://forum.ansible.com/t/trying-to-speed-up-ansible-with-serial-running-but-it-hangs-after-a-while-constantly/40370/9 "2025-02-20T23:16:24Z")

</div>

You can avoid that if you set  
`$ export ANSIBLE_HOST_KEY_CHECKING=False`

[https://docs.ansible.com/ansible/latest/inventory\_guide/connection\_details.html#managing-host-key-checking](https://docs.ansible.com/ansible/latest/inventory_guide/connection_details.html#managing-host-key-checking)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/ansible/original/1X/1882d9a01da5d8996a862359b0913cc568a39b84.svg) [@system](https://forum.ansible.com/u/system)\
**Post date:** [March 22, 2025, 11:17pm UTC](https://forum.ansible.com/t/trying-to-speed-up-ansible-with-serial-running-but-it-hangs-after-a-while-constantly/40370/10 "2025-03-22T23:17:25Z")

</div>

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.