Upgrade failed due to connection issue

Hi Team,

We are trying to upgrade our ansible awx to 0.21.0 and we are getting the connection issue.

Our configurations:

We have two awx instances in same EKS cluster

EKS cluster version: 1.21

We are using our external porstgresql.

Error: awx.main.wsbroadcast Connection from awx-76978c6c89-g7d8z to 10.38.211.35 failed: 'Cannot connect to host 10.38.211.35:8052 ssl:False [C │

Can someone help on this

Thank you,

Narender

Hi Narender,

Which pod and container is this error message originating from? Are pods / containers crashing?

AWX Team

Hi,

We are getting this error in awx-task and awx-web. The pods keeps on restarting.

Thanks,

Narender

Hi,

I’m facing the same issue where awx-web and awx-task containers keep crashing repeatedly -

awx-web logs

mounting awx.wsgi:application on /

2022-08-31 16:47:14,691 INFO exited: daphne (terminated by SIGKILL; not expected)

2022-08-31 16:47:14,691 INFO exited: daphne (terminated by SIGKILL; not expected)

2022-08-31 16:47:15,990 INFO spawned: ‘daphne’ with pid 107

2022-08-31 16:47:15,990 INFO spawned: ‘daphne’ with pid 107

DAMN ! worker 3 (pid: 100) died, killed by signal 9 :frowning: trying respawn …

Respawned uWSGI worker 3 (new pid: 108)

mounting awx.wsgi:application on /

2022-08-31 16:47:26,095 INFO success: wsbroadcast entered RUNNING state, process has stayed up for > than 30 seconds (startsecs)

2022-08-31 16:47:26,095 INFO success: wsbroadcast entered RUNNING state, process has stayed up for > than 30 seconds (startsecs)

DAMN ! worker 5 (pid: 101) died, killed by signal 9 :frowning: trying respawn …

Respawned uWSGI worker 5 (new pid: 109)

mounting awx.wsgi:application on /

awx-task logs

2022-08-31 16:50:43,979 WARN received SIGQUIT indicating exit request

2022-08-31 16:50:43,979 INFO waiting for superwatcher, callback-receiver to die

2022-08-31 16:50:43,979 INFO waiting for superwatcher, callback-receiver to die

2022-08-31 16:50:43,983 INFO stopped: callback-receiver (terminated by SIGTERM)

2022-08-31 16:50:43,983 INFO stopped: callback-receiver (terminated by SIGTERM)

2022-08-31 16:50:43,983 INFO stopped: superwatcher (terminated by SIGTERM)

2022-08-31 16:50:43,983 INFO stopped: superwatcher (terminated by SIGTERM)

Thanks,
Kaushik

Can someone please suggest a solution to this?

What are the statuses of your running pods? Does your minikube/k8s cluster have plenty of resources to run the pods? What does “kubectl describe deploy/awx -n awx” output?
Thanks,

AWX Team