# restart service, check if port is ready to accept and then move to next host

**URL:** <https://forum.ansible.com/t/restart-service-check-if-port-is-ready-to-accept-and-then-move-to-next-host/38557>\
**Category:** Ansible Project\
**Created:** [October 31, 2023, 2:08pm UTC](https://forum.ansible.com/t/restart-service-check-if-port-is-ready-to-accept-and-then-move-to-next-host/38557 "2023-10-31T14:08:09Z")\
**Posts on this page:** 17\
**Page:** 1

<div class="post-metadata">

**Author:** ![Sameer\_Modak](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/sameer_modak/32/5180_2.png) [@Sameer\_Modak](https://forum.ansible.com/u/Sameer_Modak)\
**Post date:** [October 31, 2023, 2:08pm UTC](https://forum.ansible.com/t/restart-service-check-if-port-is-ready-to-accept-and-then-move-to-next-host/38557/1 "2023-10-31T14:08:09Z")

</div>

restart service, check if service is ready to accept connection because it takes time to come up. Once we sure its listening on port then only move to next host. unless dont move because we can only afford to have one service down at a time.

is there any to short hand or ansible native way to handle this using ansible module.

code:

name: Restart zookeeper followers

throttle: 1

any\_errors\_fatal: true

shell: |

systemctl restart {{zookeeper\_service\_name}}

timeout 22 sh -c ‘until nc localhost {{zookeeper\_server\_port}}; do sleep 1; done’

when: not zkmode.stdout\_lines is search(‘leader’)

---

<div class="post-metadata">

**Author:** ![Will\_McDonald](https://avatars.discourse-cdn.com/v4/letter/w/f475e1/32.png) [@Will\_McDonald](https://forum.ansible.com/u/Will_McDonald)\
**Post date:** [October 31, 2023, 2:23pm UTC](https://forum.ansible.com/t/restart-service-check-if-port-is-ready-to-accept-and-then-move-to-next-host/38557/2 "2023-10-31T14:23:00Z")

</div>

I’d suggest reading up on rolling updates using serial:

[https://docs.ansible.com/ansible/latest/playbook\_guide/guide\_rolling\_upgrade.html#the-rolling-upgrade](https://docs.ansible.com/ansible/latest/playbook_guide/guide_rolling_upgrade.html#the-rolling-upgrade)  
[https://docs.ansible.com/ansible/latest/playbook\_guide/playbooks\_strategies.html#setting-the-batch-size-with-serial](https://docs.ansible.com/ansible/latest/playbook_guide/playbooks_strategies.html#setting-the-batch-size-with-serial)

You can use wait\_for or wait\_for\_connection to ensure service availability before continuing:

[https://docs.ansible.com/ansible/latest/collections/ansible/builtin/wait\_for\_module.html](https://docs.ansible.com/ansible/latest/collections/ansible/builtin/wait_for_module.html)  
[https://docs.ansible.com/ansible/latest/collections/ansible/builtin/wait\_for\_connection\_module.html](https://docs.ansible.com/ansible/latest/collections/ansible/builtin/wait_for_connection_module.html)

---

<div class="post-metadata">

**Author:** ![Sameer\_Modak](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/sameer_modak/32/5180_2.png) [@Sameer\_Modak](https://forum.ansible.com/u/Sameer_Modak)\
**Post date:** [October 31, 2023, 5:54pm UTC](https://forum.ansible.com/t/restart-service-check-if-port-is-ready-to-accept-and-then-move-to-next-host/38557/3 "2023-10-31T17:54:27Z")

</div>

Hello Will,

I have used throttle so that part is sorted. But i dont think wait\_for works here for example.  
task 1 restart. \<— now in this task already he has restarted all hosts one by one  
task 2 wait\_for ← this will fail if port does not come up but no use because restart is triggered.

we just want to know if in one task it restarts and checks if fails aborts play thats it. Now we got the results but used shell module.

---

<div class="post-metadata">

**Author:** ![Will\_McDonald](https://avatars.discourse-cdn.com/v4/letter/w/f475e1/32.png) [@Will\_McDonald](https://forum.ansible.com/u/Will_McDonald)\
**Post date:** [October 31, 2023, 11:58pm UTC](https://forum.ansible.com/t/restart-service-check-if-port-is-ready-to-accept-and-then-move-to-next-host/38557/4 "2023-10-31T23:58:10Z")

</div>

I don’t entirely understand your approach, constraints or end-to-end requirements here, but trying to read between the lines…

1. You have a cluster of zookeeper nodes (presumably 2n+1 so 3, 5 or more nodes)

2. You want to do a rolling restart of these nodes 1 at a time, wait for the node to come back up, check it’s functioning, and if that doesn’t work, fail the run

3. With your existing approach you can limit the restart of a service using throttle at the task level, but then don’t know how to handle failure in a subsequent task

4. You don’t think wait\_for will work because you only throttle on the restart task

(Essentially you want your condition “has the service restarted successfully” to be in the task itself.)

Again some thoughts that might help you work through this…

1. Any reason you couldn’t just use serial at a playbook level? If so, what is that?

2. If you must throttle rather than serial, consider using it in a block along with a failed\_when

3. Try and avoid using shell and use builtin constructs like service, it’ll save you longer term pain

Read through the links I posted earlier and explain what might stop you using the documented approach.

This post from Vladimir on Superuser might be useful too: [https://superuser.com/questions/1664197/ansible-keyword-throttle](https://superuser.com/questions/1664197/ansible-keyword-throttle) (loads of other 2n+1 rolling update/restart examples out there too: [https://stackoverflow.com/questions/62378317/ansible-rolling-restart-multi-cluster-environment](https://stackoverflow.com/questions/62378317/ansible-rolling-restart-multi-cluster-environment))

---

<div class="post-metadata">

**Author:** ![Will\_McDonald](https://avatars.discourse-cdn.com/v4/letter/w/f475e1/32.png) [@Will\_McDonald](https://forum.ansible.com/u/Will_McDonald)\
**Post date:** [November 1, 2023, 12:02am UTC](https://forum.ansible.com/t/restart-service-check-if-port-is-ready-to-accept-and-then-move-to-next-host/38557/5 "2023-11-01T00:02:45Z")

</div>

Edit: s/along with a failed\_when/along with wait\_for/

---

<div class="post-metadata">

**Author:** ![Sameer\_Modak](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/sameer_modak/32/5180_2.png) [@Sameer\_Modak](https://forum.ansible.com/u/Sameer_Modak)\
**Post date:** [November 1, 2023, 10:09am UTC](https://forum.ansible.com/t/restart-service-check-if-port-is-ready-to-accept-and-then-move-to-next-host/38557/6 "2023-11-01T10:09:54Z")

</div>

Let me try with block and serial and get back to you

---

<div class="post-metadata">

**Author:** ![Sameer\_Modak](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/sameer_modak/32/5180_2.png) [@Sameer\_Modak](https://forum.ansible.com/u/Sameer_Modak)\
**Post date:** [November 3, 2023, 1:22pm UTC](https://forum.ansible.com/t/restart-service-check-if-port-is-ready-to-accept-and-then-move-to-next-host/38557/7 "2023-11-03T13:22:18Z")

</div>

Hello will,

i tried to do it with block and serial no it does not work say’s block cant have serial

tasks:

- name: block check

block:

- name: run this shell

shell: ‘systemctl restart “{{zookeeper\_service\_name}}”’

- name: debug

debug:

msg: “running my task”

- name: now run this task

shell: timeout -k 3 1m sh -c ‘until nc -zv localhost {{hostvars[inventory\_hostname].zk\_port}}; do sleep 1; done’

when:

- not zkmode is search(‘leader’)

serial: 1

---

<div class="post-metadata">

**Author:** ![Will\_McDonald](https://avatars.discourse-cdn.com/v4/letter/w/f475e1/32.png) [@Will\_McDonald](https://forum.ansible.com/u/Will_McDonald)\
**Post date:** [November 3, 2023, 1:50pm UTC](https://forum.ansible.com/t/restart-service-check-if-port-is-ready-to-accept-and-then-move-to-next-host/38557/8 "2023-11-03T13:50:14Z")

</div>

I think you’ve misunderstood what I suggested. (Or I’ve explained it poorly.)

If you use serial, you wouldn’t need a block necessarily as you’d be executing over the inventory hosts one-at-a-time.

If you insist on sticking with throttle, try it with a block in order to group your service restart and service availability check.

I strongly going and taking the time to read the rolling update example that’s already documented, understand it and then think about how to apply that to what you’re trying to achieve.

---

<div class="post-metadata">

**Author:** ![Sameer\_Modak](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/sameer_modak/32/5180_2.png) [@Sameer\_Modak](https://forum.ansible.com/u/Sameer_Modak)\
**Post date:** [November 3, 2023, 3:00pm UTC](https://forum.ansible.com/t/restart-service-check-if-port-is-ready-to-accept-and-then-move-to-next-host/38557/9 "2023-11-03T15:00:21Z")

</div>

Ok my requirement is exactly the same.

[https://stackoverflow.com/questions/64048208/run-ansible-tasks-on-hosts-one-by-one](https://stackoverflow.com/questions/64048208/run-ansible-tasks-on-hosts-one-by-one)  
EXactly the same.

list of taks needs to be run one by one on single host at a time

---

<div class="post-metadata">

**Author:** ![utoddl](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/utoddl/32/249_2.png) [@utoddl](https://forum.ansible.com/u/utoddl)\
**Post date:** [November 3, 2023, 3:29pm UTC](https://forum.ansible.com/t/restart-service-check-if-port-is-ready-to-accept-and-then-move-to-next-host/38557/10 "2023-11-03T15:29:59Z")

</div>

That’s correct; serial is not a task or block key word. It’s a playbook key word.

```auto
- name: One host at a time
  hosts: ducks_in_a_row
  serial: 1
  max_fail_percentage: 0
  tasks:
    - task1
    - task2
    - task3

```

Read up on [serial](https://docs.ansible.com/ansible/latest/playbook_guide/playbooks_strategies.html#setting-the-batch-size-with-serial) and [max\_fail\_percentage](https://docs.ansible.com/ansible/latest/playbook_guide/playbooks_error_handling.html#setting-a-maximum-failure-percentage) . Blocks don’t come into it.

---

<div class="post-metadata">

**Author:** ![Sameer\_Modak](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/sameer_modak/32/5180_2.png) [@Sameer\_Modak](https://forum.ansible.com/u/Sameer_Modak)\
**Post date:** [November 9, 2023, 4:29pm UTC](https://forum.ansible.com/t/restart-service-check-if-port-is-ready-to-accept-and-then-move-to-next-host/38557/11 "2023-11-09T16:29:28Z")

</div>

Hello Todd,

I tried serial and it works but my problem is, serial works in playbook so when i write import\_playbook inside include\_task: zookeeper.yaml it fails saying u cant import playbook inside task.  
Now, How do i do it then??

ok so let me give you how i am running basically i have created role prometheus which you can find here in below my personal public repo. Role has its usual main.yml which includes tasks and i have created Restartandcheck.yml which i am unable to use because import\_playbook error if i put in zookeeper.yml file

[https://github.com/sameergithub5/prometheusrole/tree/main/prometheus](https://github.com/sameergithub5/prometheusrole/tree/main/prometheus)

---

<div class="post-metadata">

**Author:** ![Zdenek\_Pyszko](https://avatars.discourse-cdn.com/v4/letter/z/ecc23a/32.png) [@Zdenek\_Pyszko](https://forum.ansible.com/u/Zdenek_Pyszko)\
**Post date:** [November 16, 2023, 12:53pm UTC](https://forum.ansible.com/t/restart-service-check-if-port-is-ready-to-accept-and-then-move-to-next-host/38557/12 "2023-11-16T12:53:25Z")

</div>

Hello Sameer,  
my two cents here as i made a quick lookup to your repo.  
I would suggest to refactor your repo to use roles.  
You have three different playbooks referenced in main.yml, which are doing more or less the same job.  
Create a role ‘enable prometheus’ which will be dynamic enough to make decision based on input variables (zookeeper, Kafka,…)  
And one tiny role to restart the services(if needed).  
Outcome: single playbook, one prometheus role, one service mgmt(restart) role, no DRY code(dont repeat yourself), re-usable.

Dne čtvrtek 9. listopadu 2023 v 17:29:28 UTC+1 uživatel Sameer Modak napsal:

---

<div class="post-metadata">

**Author:** ![Sameer\_Modak](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/sameer_modak/32/5180_2.png) [@Sameer\_Modak](https://forum.ansible.com/u/Sameer_Modak)\
**Post date:** [November 19, 2023, 7:54am UTC](https://forum.ansible.com/t/restart-service-check-if-port-is-ready-to-accept-and-then-move-to-next-host/38557/13 "2023-11-19T07:54:59Z")

</div>

Thanks a lot Zdenek.

I got it now i have heard your comments and converted this to something closer .

[https://github.com/sameergithub5/prometheusrole/tree/main/node\_exporter\_and\_prometheus\_jmx\_exporter](https://github.com/sameergithub5/prometheusrole/tree/main/node_exporter_and_prometheus_jmx_exporter)

Can you plz spot if there is a room for an improvement .

---

<div class="post-metadata">

**Author:** ![Zdenek\_Pyszko](https://avatars.discourse-cdn.com/v4/letter/z/ecc23a/32.png) [@Zdenek\_Pyszko](https://forum.ansible.com/u/Zdenek_Pyszko)\
**Post date:** [November 24, 2023, 1:03pm UTC](https://forum.ansible.com/t/restart-service-check-if-port-is-ready-to-accept-and-then-move-to-next-host/38557/14 "2023-11-24T13:03:53Z")

</div>

yea, getting better 🙂  
Have a look to diff of my fork, what could work:  
[https://github.com/sameergithub5/prometheusrole/pull/1](https://github.com/sameergithub5/prometheusrole/pull/1)

It is still pretty raw, but it contains the idea.

Dne neděle 19. listopadu 2023 v 8:54:59 UTC+1 uživatel Sameer Modak napsal:

---

<div class="post-metadata">

**Author:** ![Evan\_Hisey](https://avatars.discourse-cdn.com/v4/letter/e/ce73a5/32.png) [@Evan\_Hisey](https://forum.ansible.com/u/Evan_Hisey)\
**Post date:** [November 25, 2023, 6:48pm UTC](https://forum.ansible.com/t/restart-service-check-if-port-is-ready-to-accept-and-then-move-to-next-host/38557/15 "2023-11-25T18:48:32Z")

</div>

Zdenek-  
Quick question on your pull request, possibly missing the obvious. I see you use loop\_control to set the outer loop variable on the roles. My understanding is the the roles would be a different namespace for the loops, so not interfere with the {{ item }} for the control loop, so was this for control clarity, or am I missing something with a namespace conflict?

---

<div class="post-metadata">

**Author:** ![Zdenek\_Pyszko](https://avatars.discourse-cdn.com/v4/letter/z/ecc23a/32.png) [@Zdenek\_Pyszko](https://forum.ansible.com/u/Zdenek_Pyszko)\
**Post date:** [November 27, 2023, 8:14am UTC](https://forum.ansible.com/t/restart-service-check-if-port-is-ready-to-accept-and-then-move-to-next-host/38557/16 "2023-11-27T08:14:15Z")

</div>

Hi Evan,  
The loop\_control part already came from Sameer, i just kept this part as i didnt want to bring another level of complexity.  
But in general, i use loop\_control pretty often, especially in some deeper structures and to enforce readability, i.e.:

- input vars structure

---

<div class="post-metadata">

**Author:** ![Zdenek\_Pyszko](https://avatars.discourse-cdn.com/v4/letter/z/ecc23a/32.png) [@Zdenek\_Pyszko](https://forum.ansible.com/u/Zdenek_Pyszko)\
**Post date:** [November 27, 2023, 8:42am UTC](https://forum.ansible.com/t/restart-service-check-if-port-is-ready-to-accept-and-then-move-to-next-host/38557/17 "2023-11-27T08:42:07Z")

</div>

Regarding interference topic, this is how looping over the role without loop\_var could look like. We all see its gonna be KABOOM 🙂

- main.yml