# Increasing Replicas from 1 to 2 Fails

**URL:** <https://forum.ansible.com/t/increasing-replicas-from-1-to-2-fails/37450>\
**Category:** AWX Project\
**Tags:** awx\
**Created:** [December 9, 2022, 9:18pm UTC](https://forum.ansible.com/t/increasing-replicas-from-1-to-2-fails/37450 "2022-12-09T21:18:20Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![bmorris53](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/bmorris53/32/9924_2.png) [@bmorris53](https://forum.ansible.com/u/bmorris53)\
**Post date:** [December 9, 2022, 9:18pm UTC](https://forum.ansible.com/t/increasing-replicas-from-1-to-2-fails/37450/1 "2022-12-09T21:18:20Z")

</div>

AWX version: 21.9.0  
awx-operator version: 1.1.0

I have awx-operator deployed in a k3s multi-master cluster. I increased the # of replicas in the awx CRD from 1 to 2. awx deployment and replicaset correctly get increased.

the new awx pod gets scheduled but fails to init.

Attached are the awx-manager logs. Those seem to indicate a problem with the “Apply deployment resources” TASK.

kubectl events for the failed pod in the same attachement.

My PVCs are also set to RWX.

> **(attachments)**
>
> [troubleshooting\_replica\_Increase.txt](https://forum.ansible.com/uploads/short-url/mjX9mL7P6SsJcIvwkF0xhgGtNR4.txt) (52.6 KB)

---

<div class="post-metadata">

**Author:** ![AWX\_Project](https://avatars.discourse-cdn.com/v4/letter/a/eada6e/32.png) [@AWX\_Project](https://forum.ansible.com/u/AWX_Project)\
**Post date:** [December 14, 2022, 7:46pm UTC](https://forum.ansible.com/t/increasing-replicas-from-1-to-2-fails/37450/2 "2022-12-14T19:46:04Z")

</div>

Hello,  
It sounds like the project’s PVC may be the culprit. Project persistence is not required. When the pods come back up they will pull your projects anew from SCM. Perhaps try reconfiguring this to not require project persistence and see if that helps. Please let us know if it does.

This does sound like it may be a bug with the project’s persistence feature. Is there an issue associated with this report?

-AWX Team

---

<div class="post-metadata">

**Author:** ![bmorris53](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/bmorris53/32/9924_2.png) [@bmorris53](https://forum.ansible.com/u/bmorris53)\
**Post date:** [December 15, 2022, 2:55am UTC](https://forum.ansible.com/t/increasing-replicas-from-1-to-2-fails/37450/3 "2022-12-15T02:55:17Z")

</div>

Thank you for the suggestion! I will give this a go in the morning and will report back.

---

<div class="post-metadata">

**Author:** ![bmorris53](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/bmorris53/32/9924_2.png) [@bmorris53](https://forum.ansible.com/u/bmorris53)\
**Post date:** [December 15, 2022, 7:00pm UTC](https://forum.ansible.com/t/increasing-replicas-from-1-to-2-fails/37450/4 "2022-12-15T19:00:54Z")

</div>

Changing project persistence to false did the trick. I now have two instance deployed and both are reporting healthy when looking at them from within the AWX GUI.

The only thing I am seeing now is that the PVC for the Postgres pod is complaining about a failed mount:

Warning FailedMount 60s (x487 over 16h) kubelet MountVolume.MountDevice failed for volume “pvc-790c4558-4718-4a5f-93ac-df7fa8383988” : rpc error: code = FailedPrecondition desc = volume pvc-790c4558-4718-4a5f-93ac-df7fa8383988 requires shared access but is not marked for shared use

I don’t seem to be seeing any adverse affects from the above. The GUI is still accessible, controlplane jobs have ran on both instances. Normal jobs being executed in the default ContainerGroup are working just fine.

Do I just ignore this or is something just waiting to leap out and bite me?

Thank you for the help!

---

<div class="post-metadata">

**Author:** ![AWX\_Project](https://avatars.discourse-cdn.com/v4/letter/a/eada6e/32.png) [@AWX\_Project](https://forum.ansible.com/u/AWX_Project)\
**Post date:** [December 16, 2022, 6:14pm UTC](https://forum.ansible.com/t/increasing-replicas-from-1-to-2-fails/37450/5 "2022-12-16T18:14:54Z")

</div>

yeah probably can ignore the warning. wondering what “kubectl describe pvc/pvc-790c4558-4718-4a5f-93ac-df7fa8383988” returns? what is this pvc intended for?

---

<div class="post-metadata">

**Author:** ![bmorris53](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/bmorris53/32/9924_2.png) [@bmorris53](https://forum.ansible.com/u/bmorris53)\
**Post date:** [December 19, 2022, 7:17pm UTC](https://forum.ansible.com/t/increasing-replicas-from-1-to-2-fails/37450/6 "2022-12-19T19:17:08Z")

</div>

This is the postgres-13 PVC.

There are no events when I look under the describe for both the PV and the PVC.

PVC and PV output attached.

> **(attachments)**
>
> [PVC\_Troubleshooting.txt](https://forum.ansible.com/uploads/short-url/pZUR7i7xRYwrf2DQLdEZzs6WfFC.txt) (2.01 KB)

---

<div class="post-metadata">

**Author:** ![AWX\_Project](https://avatars.discourse-cdn.com/v4/letter/a/eada6e/32.png) [@AWX\_Project](https://forum.ansible.com/u/AWX_Project)\
**Post date:** [December 21, 2022, 7:58pm UTC](https://forum.ansible.com/t/increasing-replicas-from-1-to-2-fails/37450/7 "2022-12-21T19:58:42Z")

</div>

although benign, the warning might be something we wish to fix. would you mind opening an issue in awx-operator for this? thanks!

AWX Team
