AWX Operator on GKE Autopilot

Hi,

I’m trying to deploy AWX operator on GKE autopilot, following this instructions: https://github.com/ansible/awx-operator/blob/devel/.helm/starter/README.md.

The operator gets deployment, however when I create a new AWX instace, the operator keeps restarting due to a CrashLoopBackOff.

The logs from AWX operator seems to give no hints about the problem:

{"level":"info","ts":"2024-04-01T10:07:54Z","logger":"cmd","msg":"Version","Go Version":"go1.20.12","GOOS":"linux","GOARCH":"amd64","ansible-operator":"v1.34.0","commit":"d26c43bf94...be33190fb"}
{"level":"info","ts":"2024-04-01T10:07:54Z","logger":"cmd","msg":"Watching namespaces","namespaces":["awx"]}
{"level":"info","ts":"2024-04-01T10:07:54Z","logger":"watches","msg":"Environment variable not set; using default value","envVar":"ANSIBLE_VERBOSITY_AWX_AWX_ANSIBLE_COM","default":2}
{"level":"info","ts":"2024-04-01T10:07:54Z","logger":"watches","msg":"Environment variable not set; using default value","envVar":"ANSIBLE_VERBOSITY_AWXBACKUP_AWX_ANSIBLE_COM","default":2}
{"level":"info","ts":"2024-04-01T10:07:54Z","logger":"watches","msg":"Environment variable not set; using default value","envVar":"ANSIBLE_VERBOSITY_AWXRESTORE_AWX_ANSIBLE_COM","default":2}
{"level":"info","ts":"2024-04-01T10:07:54Z","logger":"watches","msg":"Environment variable not set; using default value","envVar":"ANSIBLE_VERBOSITY_AWXMESHINGRESS_AWX_ANSIBLE_COM","default":2}
{"level":"info","ts":"2024-04-01T10:07:54Z","logger":"ansible-controller","msg":"Watching resource","Options.Group":"awx.ansible.com","Options.Version":"v1beta1","Options.Kind":"AWX"}
{"level":"info","ts":"2024-04-01T10:07:54Z","logger":"ansible-controller","msg":"Watching resource","Options.Group":"awx.ansible.com","Options.Version":"v1beta1","Options.Kind":"AWXBackup"}
{"level":"info","ts":"2024-04-01T10:07:54Z","logger":"ansible-controller","msg":"Watching resource","Options.Group":"awx.ansible.com","Options.Version":"v1beta1","Options.Kind":"AWXRestore"}
{"level":"info","ts":"2024-04-01T10:07:54Z","logger":"ansible-controller","msg":"Watching resource","Options.Group":"awx.ansible.com","Options.Version":"v1alpha1","Options.Kind":"AWXMeshIngress"}
{"level":"info","ts":"2024-04-01T10:07:54Z","logger":"proxy","msg":"Starting to serve","Address":"127.0.0.1:8888"}
I0401 10:07:54.174711       7 leaderelection.go:250] attempting to acquire leader lease awx/awx-operator...
{"level":"info","ts":"2024-04-01T10:07:54Z","logger":"apiserver","msg":"Starting to serve metrics listener","Address":"localhost:5050"}
{"level":"info","ts":"2024-04-01T10:07:54Z","msg":"starting server","kind":"health probe","addr":"[::]:6789"}
{"level":"info","ts":"2024-04-01T10:07:54Z","logger":"controller-runtime.metrics","msg":"Starting metrics server"}
{"level":"info","ts":"2024-04-01T10:07:54Z","logger":"controller-runtime.metrics","msg":"Serving metrics server","bindAddress":"127.0.0.1:8080","secure":false}
I0401 10:08:12.139338       7 leaderelection.go:260] successfully acquired lease awx/awx-operator
{"level":"info","ts":"2024-04-01T10:08:12Z","msg":"Starting EventSource","controller":"awx-controller","source":"kind source: *unstructured.Unstructured"}
{"level":"info","ts":"2024-04-01T10:08:12Z","msg":"Starting Controller","controller":"awx-controller"}
{"level":"info","ts":"2024-04-01T10:08:12Z","msg":"Starting EventSource","controller":"awxrestore-controller","source":"kind source: *unstructured.Unstructured"}
{"level":"info","ts":"2024-04-01T10:08:12Z","msg":"Starting Controller","controller":"awxrestore-controller"}
{"level":"info","ts":"2024-04-01T10:08:12Z","msg":"Starting EventSource","controller":"awxmeshingress-controller","source":"kind source: *unstructured.Unstructured"}
{"level":"info","ts":"2024-04-01T10:08:12Z","msg":"Starting Controller","controller":"awxmeshingress-controller"}
{"level":"info","ts":"2024-04-01T10:08:12Z","msg":"Starting EventSource","controller":"awxbackup-controller","source":"kind source: *unstructured.Unstructured"}
{"level":"info","ts":"2024-04-01T10:08:12Z","msg":"Starting Controller","controller":"awxbackup-controller"}

Any ideas?

can you try kubectl logs -p to get the log from the previously crashed container