Unable to create AWX Operator Resource

Get Help
, , ,
1 
Hi all. Does anyone have experience with the AWX Operator? I'm migrating from 17.1.0 and have moved my postgres database to an external cluster.

I've created secrets for my postgres configuration, admin password, secret key and TLS cert. However, when I try to use the following it doesn't create all the pods:

apiVersion: awx.ansible.com/v1beta1
kind: AWX
metadata:
  name: awx
spec:
  old_postgres_configuration_secret: awx-old-postgres-configuration
  admin_password_secret: awx-admin-password
  secret_key_secret: awx-secret-key
  service_type: nodeport
  hostname: awx.nl.mdb-lab.com
  ingress_type: ingress
  ingress_class_name: avi-lb
  ingress_tls_secret: awx.nl.mdb-lab.com

The only pod I get is the controller manager "awx-operator-controller-manager-6ddc7745d9-7xqhg".

Any idea what I'm missing?
2

Are there any errors in the operator logs? Here are instructions on how to check:

Also, what namespace is your awx resource being created in?

kubectl get awx -A
1 Like
3

Hi Christian,

I’m deploying into the “awx” namespace.

Very odd. When I run “k logs -f deployments/awx-operator-controller-manager -c awx-manager -n awx”

All I get is:

{"level":"info","ts":"2024-01-17T07:37:31Z","logger":"cmd","msg":"Version","Go Version":"go1.19.13","GOOS":"linux","GOARCH":"amd64","ansible-operator":"v1.32.0","commit":"7bce7c2edbb6cba9e77819209cd813a3d5d09f0f"}
{"level":"info","ts":"2024-01-17T07:37:31Z","logger":"cmd","msg":"Watching single namespace.","Namespace":"awx"}
I0117 07:37:32.694554       7 request.go:690] Waited for 1.036428446s due to client-side throttling, not priority and fairness, request: GET:https://10.96.0.1:443/apis/system.antrea.io/v1beta1?timeout=32s
{"level":"info","ts":"2024-01-17T07:37:32Z","logger":"controller-runtime.metrics","msg":"Metrics server is starting to listen","addr":"127.0.0.1:8080"}
{"level":"info","ts":"2024-01-17T07:37:32Z","logger":"watches","msg":"Environment variable not set; using default value","envVar":"ANSIBLE_VERBOSITY_AWX_AWX_ANSIBLE_COM","default":2}
{"level":"info","ts":"2024-01-17T07:37:32Z","logger":"watches","msg":"Environment variable not set; using default value","envVar":"ANSIBLE_VERBOSITY_AWXBACKUP_AWX_ANSIBLE_COM","default":2}
{"level":"info","ts":"2024-01-17T07:37:32Z","logger":"watches","msg":"Environment variable not set; using default value","envVar":"ANSIBLE_VERBOSITY_AWXRESTORE_AWX_ANSIBLE_COM","default":2}
{"level":"info","ts":"2024-01-17T07:37:32Z","logger":"ansible-controller","msg":"Watching resource","Options.Group":"awx.ansible.com","Options.Version":"v1beta1","Options.Kind":"AWX"}
{"level":"info","ts":"2024-01-17T07:37:32Z","logger":"ansible-controller","msg":"Watching resource","Options.Group":"awx.ansible.com","Options.Version":"v1beta1","Options.Kind":"AWXBackup"}
{"level":"info","ts":"2024-01-17T07:37:32Z","logger":"ansible-controller","msg":"Watching resource","Options.Group":"awx.ansible.com","Options.Version":"v1beta1","Options.Kind":"AWXRestore"}
{"level":"info","ts":"2024-01-17T07:37:32Z","logger":"proxy","msg":"Starting to serve","Address":"127.0.0.1:8888"}
{"level":"info","ts":"2024-01-17T07:37:32Z","logger":"apiserver","msg":"Starting to serve metrics listener","Address":"localhost:5050"}
{"level":"info","ts":"2024-01-17T07:37:32Z","msg":"Starting server","kind":"health probe","addr":"[::]:6789"}
{"level":"info","ts":"2024-01-17T07:37:32Z","msg":"Starting server","path":"/metrics","kind":"metrics","addr":"127.0.0.1:8080"}
I0117 07:37:32.741143       7 leaderelection.go:248] attempting to acquire leader lease awx/awx-operator...
I0117 07:37:49.135707       7 leaderelection.go:258] successfully acquired lease awx/awx-operator
{"level":"info","ts":"2024-01-17T07:37:49Z","msg":"Starting EventSource","controller":"awxrestore-controller","source":"kind source: *unstructured.Unstructured"}
{"level":"info","ts":"2024-01-17T07:37:49Z","msg":"Starting Controller","controller":"awxrestore-controller"}
{"level":"info","ts":"2024-01-17T07:37:49Z","msg":"Starting EventSource","controller":"awx-controller","source":"kind source: *unstructured.Unstructured"}
{"level":"info","ts":"2024-01-17T07:37:49Z","msg":"Starting Controller","controller":"awx-controller"}
{"level":"info","ts":"2024-01-17T07:37:49Z","msg":"Starting EventSource","controller":"awxbackup-controller","source":"kind source: *unstructured.Unstructured"}
{"level":"info","ts":"2024-01-17T07:37:49Z","msg":"Starting Controller","controller":"awxbackup-controller"}
{"level":"info","ts":"2024-01-17T07:37:49Z","msg":"Starting workers","controller":"awx-controller","worker count":4}
{"level":"info","ts":"2024-01-17T07:37:49Z","msg":"Starting workers","controller":"awxbackup-controller","worker count":4}
{"level":"info","ts":"2024-01-17T07:37:49Z","msg":"Starting workers","controller":"awxrestore-controller","worker count":4}

Very odd!

4

Hello @virtualhobbit thank you for your reply! Would you mind providing us with the output of kubectl get awx -A? I don’t see a namespace specified on your AWX CR yaml we just want to rule out that it may be in the wrong namespace.

1 Like
5

FYI, re: the awx-operator pod logs, that confirms that the operator itself is up and running in the awx namespace and it watching for new AWX resources in that namespace. It also tells us that it has not seen an AWX custom resource be created in the awx namespace on that cluster.

6

Christian’s “k get awx -A” put me on the right track. I was literally deploying to the default namespace, and now I’m dying with shame! :frowning:

It’s now deploying an awx-postgres-13-0 pod, despite being given an “awx-old-postgres-configuration” config.

The quest continues…

7

No worries, I’ve done the same thing before myself :slight_smile:

As for the database, if you do not specify your external postgres db creds with the postgres_configuration_secret, it will will default to creating a new managed postgres pod.

There are two ways to accomplish what you are trying to do:

Option 1: specify both the old_postgres_configuration_secret as well as the postgres_configuration_secret on the AWX spec of your new instance.

Option 2: don’t specify old_postgres_configuration_secret, only specify the postgres_configuration_secret since your external database already has all of the resources.

In either case, you will need to specify the secret_key_secret with the value from the original deployment. Docs here:

1 Like
8

Ah, this is where I’m going wrong. I’ve defined the following secret:

apiVersion: v1
kind: Secret
metadata:
name: awx-old-postgres-configuration
namespace: awx
stringData:
host: db.mdb-lab.com
port: “5000”
database: “awx”
username: “awx”
password: “awxpass”
type: Opaque

And:

spec:
old_postgres_configuration_secret: awx-old-postgres-configuration

You’re saying (in option 1) I guess that I need:

spec:
old_postgres_configuration_secret: awx-old-postgres-configuration
postgres_configuration_secret: awx-old-postgres-configuration

Is that correct?

Also, for the secret_key_secret, which is of the following is correct?

stringData:
secret_key: “1hdaaSBac0sRjvOmGrfrUFXY3E68iuG+oXx61m3I”

Or:

data:
secret_key: [base64-encoded-secret-key]

Many thanks for your help on this!