# awx deployment failed in k3s cluster

**URL:** <https://forum.ansible.com/t/awx-deployment-failed-in-k3s-cluster/37407>\
**Category:** AWX Project\
**Tags:** awx, ee, kubernetes\
**Created:** [December 1, 2022, 11:44am UTC](https://forum.ansible.com/t/awx-deployment-failed-in-k3s-cluster/37407 "2022-12-01T11:44:06Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ahilan\_Swaminathan](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/ahilan_swaminathan/32/4425_2.png) [@Ahilan\_Swaminathan](https://forum.ansible.com/u/Ahilan_Swaminathan)\
**Post date:** [December 1, 2022, 11:44am UTC](https://forum.ansible.com/t/awx-deployment-failed-in-k3s-cluster/37407/1 "2022-12-01T11:44:06Z")

</div>

Hi Team

When I try to deploy the awx in my k3s cluster, awx pod not started and failed to start and got “CrashLoopBackOff”. when I describe the pod I got the event as below.

Events:  
Type Reason Age From Message

* * *

Normal Scheduled 39m default-scheduler Successfully assigned awx/awx-78ccbd4b9b-r7lsc to xl8phxc360awxtst  
Normal Pulled 39m kubelet Container image “[quay.io/ansible/awx-ee:latest](http://quay.io/ansible/awx-ee:latest)” already present on machine  
Normal Created 39m kubelet Created container init  
Normal Started 39m kubelet Started container init  
Normal Pulled 39m kubelet Container image “[docker.io/redis:7](http://docker.io/redis:7)” already present on machine  
Normal Created 39m kubelet Created container redis  
Normal Started 39m kubelet Started container redis  
Normal Pulled 39m kubelet Container image “[quay.io/ansible/awx:21.8.0](http://quay.io/ansible/awx:21.8.0)” already present on machine  
Normal Created 39m kubelet Created container awx-task  
Normal Started 39m kubelet Started container awx-task  
Normal Pulled 39m kubelet Container image “[quay.io/ansible/awx-ee:latest](http://quay.io/ansible/awx-ee:latest)” already present on machine  
Normal Created 39m kubelet Created container awx-ee  
Normal Started 39m kubelet Started container awx-ee  
Normal Created 38m (x3 over 39m) kubelet Created container awx-web  
Normal Started 38m (x3 over 39m) kubelet Started container awx-web  
Normal Pulled 37m (x4 over 39m) kubelet Container image “[quay.io/ansible/awx:21.8.0](http://quay.io/ansible/awx:21.8.0)” already present on machine  
**Warning BackOff 4m40s (x139 over 38m) kubelet Back-off restarting failed container**

My awx status as below!!!

[kubeadmin@xl8phxc360awxtst ~]$ kubectl -n awx get awx,all,ingress,secrets  
NAME AGE  
[awx.awx.ansible.com/awx](http://awx.awx.ansible.com/awx) 81m

NAME READY STATUS RESTARTS AGE  
pod/awx-operator-controller-manager-585df5d96d-hk2xs 2/2 Running 0 94m  
pod/awx-postgres-13-0 1/1 Running 0 47m  
**pod/awx-78ccbd4b9b-r7lsc 3/4 CrashLoopBackOff 12 (4m55s ago) 47m**

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE  
service/awx-operator-controller-manager-metrics-service ClusterIP xx.xx.xx.xx 8443/TCP 94m  
service/awx-postgres-13 ClusterIP None 5432/TCP 80m  
service/awx-service ClusterIP xx.xx.xx.xx 80/TCP 78m

NAME READY UP-TO-DATE AVAILABLE AGE  
deployment.apps/awx-operator-controller-manager 1/1 1 1 94m  
deployment.apps/awx 0/1 1 0 78m

NAME DESIRED CURRENT READY AGE  
replicaset.apps/awx-operator-controller-manager-585df5d96d 1 1 1 94m  
replicaset.apps/awx-78ccbd4b9b 1 1 0 78m

NAME READY AGE  
statefulset.apps/awx-postgres-13 1/1 80m

NAME CLASS HOSTS ADDRESS PORTS AGE  
[ingress.networking.k8s.io/awx-ingress](http://ingress.networking.k8s.io/awx-ingress) traefik [xl8phxc360awxtst.ipengine360s1.net](http://xl8phxc360awxtst.ipengine360s1.net) xx.xx.xx.xx 80, 443 78m

NAME TYPE DATA AGE  
secret/awx-admin-password Opaque 1 81m  
secret/awx-postgres-configuration Opaque 6 81m  
secret/awx-secret-tls [kubernetes.io/tls](http://kubernetes.io/tls) 2 81m  
secret/redhat-operators-pull-secret Opaque 1 81m  
secret/awx-receptor-ca [kubernetes.io/tls](http://kubernetes.io/tls) 2 78m  
secret/awx-receptor-work-signing Opaque 2 78m  
secret/awx-app-credentials Opaque 3 78m  
secret/awx-secret-key Opaque 1 80m  
secret/awx-broadcast-websocket Opaque 1 80m  
[kubeadmin@xl8phxc360awxtst ~]$

Please suggest and hel;p me to resolve the issue

---

<div class="post-metadata">

**Author:** ![AWX\_Project](https://avatars.discourse-cdn.com/v4/letter/a/eada6e/32.png) [@AWX\_Project](https://forum.ansible.com/u/AWX_Project)\
**Post date:** [December 2, 2022, 6:24pm UTC](https://forum.ansible.com/t/awx-deployment-failed-in-k3s-cluster/37407/2 "2022-12-02T18:24:27Z")

</div>

sounds like it could be due to insufficient resources to start the container. Does bumping available resources help get the pod started?

take a look at this guide [https://komodor.com/learn/how-to-fix-crashloopbackoff-kubernetes-error/](https://komodor.com/learn/how-to-fix-crashloopbackoff-kubernetes-error/)

AWX Team

---

<div class="post-metadata">

**Author:** ![Ahilan\_Swaminathan](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/ahilan_swaminathan/32/4425_2.png) [@Ahilan\_Swaminathan](https://forum.ansible.com/u/Ahilan_Swaminathan)\
**Post date:** [December 13, 2022, 5:51am UTC](https://forum.ansible.com/t/awx-deployment-failed-in-k3s-cluster/37407/3 "2022-12-13T05:51:37Z")

</div>

Hi Team

We have 12GB free RAM and our server having 8vcpu and we have enough free space in all filesystem including root. Not sure where/what we bump in resources.

---

<div class="post-metadata">

**Author:** ![Wei-Yen\_Tan](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/wei-yen_tan/32/4111_2.png) [@Wei-Yen\_Tan](https://forum.ansible.com/u/Wei-Yen_Tan)\
**Post date:** [December 13, 2022, 8:05am UTC](https://forum.ansible.com/t/awx-deployment-failed-in-k3s-cluster/37407/4 "2022-12-13T08:05:14Z")

</div>

Please describe the pod so we can see further errors

---

<div class="post-metadata">

**Author:** ![Ahilan\_Swaminathan](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/ahilan_swaminathan/32/4425_2.png) [@Ahilan\_Swaminathan](https://forum.ansible.com/u/Ahilan_Swaminathan)\
**Post date:** [December 16, 2022, 4:21pm UTC](https://forum.ansible.com/t/awx-deployment-failed-in-k3s-cluster/37407/5 "2022-12-16T16:21:03Z")

</div>

Hi Yen

Here attached output of pod describe and container status

> **(attachments)**
>
> [pods\_containers\_details.txt](https://forum.ansible.com/uploads/short-url/AhCkJzgmURsK0hBjMG8i1JEEWLZ.txt) (15.2 KB)

---

<div class="post-metadata">

**Author:** ![AWX\_Project](https://avatars.discourse-cdn.com/v4/letter/a/eada6e/32.png) [@AWX\_Project](https://forum.ansible.com/u/AWX_Project)\
**Post date:** [January 13, 2023, 6:27pm UTC](https://forum.ansible.com/t/awx-deployment-failed-in-k3s-cluster/37407/6 "2023-01-13T18:27:59Z")

</div>

Hi were you able to resolve this issue? If possible can you provide logging details emitted from the awx-web container (if it starts at all)? you can use tool like stern to aggregate all logs emitted from all pods in the awx namespace. With additional info we can probably help figure out the root cause to the crash backoff.

Thanks!

AWX Team
