I’m not the most experienced with this stuff but I have a cluster and a couple of nodes up and running. Im connected and using kubectl commands.
I’ve been googling and using chatgpt but still cant get a good set of instructions.
NAME READY STATUS RESTARTS AGE
awx-operator-controller-manager-696bd9655-qgqz7 2/2 Running 0 27m
I do, but I use a helm chart rather than the operator!
I’d expect the operator logs to have more details about where it’s going wrong, most obvious causes being the operator not having the right RBAC permissions to create the underlying kubernetes resources.
kubectl logs awx-operator-controller-manager-696bd9655-qgqz7
Thanks for the response. Any chance you point me towards what instructions you followed to get it working?
Hi,
Ive used the operator to deploy to EKS.
Did you check the operator pod logs? They may tell you why things are not progressing. You should also check the deployments and see if the awx deployments have been created
Heres the main steps from my playbook
- name: Checkout AWX
ansible.builtin.git:
repo: <awx repo url>
dest: /tmp/awx
version: "{{ awx_operator_version }}"
- name: Replace Kubectl Apply
ansible.builtin.replace:
path: /tmp/awx/Makefile
regexp: \| kubectl apply -f -
replace: '> /tmp/awx-operator.yaml'
- name: Run make
ansible.builtin.command: make deploy
args:
chdir: /tmp/awx
environment:
NAMESPACE: "{{ awx_namespace }}"
- name: Apply Manifests
kubernetes.core.k8s:
state: present
namespace: "{{ awx_namespace }}"
template:
- /tmp/awx-operator.yaml
- my/custom/resources.yml
my/custom/resources.yml contains the extra inputs described in the repos docs. Like the admin password secret, the ingress setup, postgres setup, etc
--------------------------- Ansible Task StdOut -------------------------------
TASK [backup : Create management pod from templated deployment config] *********
task path: /opt/ansible/roles/backup/tasks/init.yml:80
-------------------------------------------------------------------------------
{"level":"info","ts":"2024-07-01T19:21:33Z","logger":"proxy","msg":"Cache miss: /v1, Kind=Pod, awx/example-awx-backup-db-management"}
{"level":"info","ts":"2024-07-01T19:21:33Z","logger":"proxy","msg":"Injecting owner reference"}
{"level":"info","ts":"2024-07-01T19:21:33Z","logger":"proxy","msg":"Read object from cache","resource":{"IsResourceRequest":true,"Path":"/api/v1/namespaces/awx/pods/example-awx-backup-db-management","Verb":"get","APIPrefix":"api","APIGroup":"","APIVersion":"v1","Namespace":"awx","Resource":"pods","Subresource":"","Name":"example-awx-backup-db-management","Parts":["pods","example-awx-backup-db-management"]}}
{"level":"info","ts":"2024-07-01T19:21:37Z","logger":"proxy","msg":"cache miss: /v1, Kind=PodList err-Index with name field:status.phase does not exist"}
{"level":"info","ts":"2024-07-01T19:21:38Z","logger":"proxy","msg":"Read object from cache","resource":{"IsResourceRequest":true,"Path":"/api/v1/namespaces/awx/pods/example-awx-backup-db-management","Verb":"get","APIPrefix":"api","APIGroup":"","APIVersion":"v1","Namespace":"awx","Resource":"pods","Subresource":"","Name":"example-awx-backup-db-management","Parts":["pods","example-awx-backup-db-management"]}}
{"level":"info","ts":"2024-07-01T19:21:43Z","logger":"proxy","msg":"Read object from cache","resource":{"IsResourceRequest":true,"Path":"/api/v1/namespaces/awx/pods/example-awx-backup-db-management","Verb":"get","APIPrefix":"api","APIGroup":"","APIVersion":"v1","Namespace":"awx","Resource":"pods","Subresource":"","Name":"example-awx-backup-db-management","Parts":["pods","example-awx-backup-db-management"]}}
{"level":"info","ts":"2024-07-01T19:21:44Z","logger":"proxy","msg":"cache miss: /v1, Kind=PodList err-Index with name field:status.phase does not exist"}
{"level":"info","ts":"2024-07-01T19:21:48Z","logger":"proxy","msg":"Read object from cache","resource":{"IsResourceRequest":true,"Path":"/api/v1/namespaces/awx/pods/example-awx-backup-db-management","Verb":"get","APIPrefix":"api","APIGroup":"","APIVersion":"v1","Namespace":"awx","Resource":"pods","Subresource":"","Name":"example-awx-backup-db-management","Parts":["pods","example-awx-backup-db-management"]}}
{"level":"info","ts":"2024-07-01T19:21:50Z","logger":"proxy","msg":"cache miss: /v1, Kind=PodList err-Index with name field:status.phase does not exist"}
{"level":"info","ts":"2024-07-01T19:21:53Z","logger":"proxy","msg":"Read object from cache","resource":{"IsResourceRequest":true,"Path":"/api/v1/namespaces/awx/pods/example-awx-backup-db-management","Verb":"get","APIPrefix":"api","APIGroup":"","APIVersion":"v1","Namespace":"awx","Resource":"pods","Subresource":"","Name":"example-awx-backup-db-management","Parts":["pods","example-awx-backup-db-management"]}}
{"level":"info","ts":"2024-07-01T19:21:56Z","logger":"proxy","msg":"cache miss: /v1, Kind=PodList err-Index with name field:status.phase does not exist"}
{"level":"info","ts":"2024-07-01T19:21:58Z","logger":"proxy","msg":"Read object from cache","resource":{"IsResourceRequest":true,"Path":"/api/v1/namespaces/awx/pods/example-awx-backup-db-management","Verb":"get","APIPrefix":"api","APIGroup":"","APIVersion":"v1","Namespace":"awx","Resource":"pods","Subresource":"","Name":"example-awx-backup-db-management","Parts":["pods","example-awx-backup-db-management"]}}
{"level":"info","ts":"2024-07-01T19:22:02Z","logger":"proxy","msg":"cache miss: /v1, Kind=PodList err-Index with name field:status.phase does not exist"}
{"level":"info","ts":"2024-07-01T19:22:03Z","logger":"proxy","msg":"Read object from cache","resource":{"IsResourceRequest":true,"Path":"/api/v1/namespaces/awx/pods/example-awx-backup-db-management","Verb":"get","APIPrefix":"api","APIGroup":"","APIVersion":"v1","Namespace":"awx","Resource":"pods","Subresource":"","Name":"example-awx-backup-db-management","Parts":["pods","example-awx-backup-db-management"]}}
{"level":"info","ts":"2024-07-01T19:22:08Z","logger":"proxy","msg":"Read object from cache","resource":{"IsResourceRequest":true,"Path":"/api/v1/namespaces/awx/pods/example-awx-backup-db-management","Verb":"get","APIPrefix":"api","APIGroup":"","APIVersion":"v1","Namespace":"awx","Resource":"pods","Subresource":"","Name":"example-awx-backup-db-management","Parts":["pods","example-awx-backup-db-management"]}}
{"level":"info","ts":"2024-07-01T19:22:08Z","logger":"proxy","msg":"cache miss: /v1, Kind=PodList err-Index with name field:status.phase does not exist"}
{"level":"info","ts":"2024-07-01T19:22:13Z","logger":"proxy","msg":"Read object from cache","resource":{"IsResourceRequest":true,"Path":"/api/v1/namespaces/awx/pods/example-awx-backup-db-management","Verb":"get","APIPrefix":"api","APIGroup":"","APIVersion":"v1","Namespace":"awx","Resource":"pods","Subresource":"","Name":"example-awx-backup-db-management","Parts":["pods","example-awx-backup-db-management"]}}
{"level":"info","ts":"2024-07-01T19:22:14Z","logger":"proxy","msg":"cache miss: /v1, Kind=PodList err-Index with name field:status.phase does not exist"}
{"level":"info","ts":"2024-07-01T19:22:18Z","logger":"proxy","msg":"Read object from cache","resource":{"IsResourceRequest":true,"Path":"/api/v1/namespaces/awx/pods/example-awx-backup-db-management","Verb":"get","APIPrefix":"api","APIGroup":"","APIVersion":"v1","Namespace":"awx","Resource":"pods","Subresource":"","Name":"example-awx-backup-db-management","Parts":["pods","example-awx-backup-db-management"]}}
{"level":"info","ts":"2024-07-01T19:22:20Z","logger":"proxy","msg":"cache miss: /v1, Kind=PodList err-Index with name field:status.phase does not exist"}
Here’s the end of the log. I think i tried to combine methods to get this installed. I’ve really confused myself.
NAME READY STATUS RESTARTS AGE
awx-demo-postgres-13-0 0/1 Pending 0 4d22h
awx-operator-controller-manager-77d65fbbc6-28cqv 2/2 Running 0 4d22h
example-awx-backup-db-management 0/1 Pending 0 78s
example-awx-mesh-ingress-c8b7bf944-xpjp2 0/1 ContainerCreating 0 4d22h
example-awx-postgres-13-0 0/1 Pending 0 4d22h
example-awx-postgres-15-0 0/1 Pending 0 4d22h
Yea, that does look like some extra stuff. Its probably best to restart from a clean slate, or at least delete everything except the operator and then restart the operator deployment (scale to 0 and then scale back to 1) so it recreates what you need.
You can inspect the other pods and see if theres a reason they are stuck in pending. They could be missing resources (pv, cpu, etc) or something.
Since your on EKS you may also want to check the nodes your using have room for more containers. I only know how to check it via the AWS gui, but when the container limit is reached new pods cannot be scheduled. You would need a larger instance
The operator logs youve shown indicate its working properly. The deployment its waiting on example-awx-backup-db-management is just not running yet. So you need to figure out why that is stuck in pending.
Ill try to start fresh and keep this alive. Might be next sprint. 
Thank youuuu