AWX 21.0.0. SSL Certificate Update/Renew

I’ve been searching high and low on how to update TLS/SSL certificates (post AWX deployment) for AWX I haven’t come across a guide on this topic.

These are the pods running:

$ kubectl get pods -n awx
NAME                                               READY   STATUS    RESTARTS        AGE
awx-registry                                       1/1     Running   5 (327d ago)    723d
awx-postgres-0                                     1/1     Running   5 (327d ago)    723d
awx-registry-pr                                    1/1     Running   4 (327d ago)    617d
awx-55ccbf9f66-vdqfz                               4/4     Running   20 (327d ago)   723d
awx-operator-controller-manager-675865446d-cs7vf   2/2     Running   15 (327d ago)   724d

Any and all help would be greatly appreciated! Thank you in advance!

If you installed via the most excellent awx-on-k3, add your tls.{crt,key} to the base/ directory, as you might have done during installation, and from its parent directory run

kubectl apply -k base

I presume that for other install methods the procedure is similar.

Hi, the key is, and what @jpmens stated is, updating the tls.crt and tls.key of the Secret resource that used by your Ingress resouce.

If the certificates you are referring to are not the one used by Ingress resouce, please tell me which one you are referring to.

Thank you. I found the base directory can I can see the previous CSR. However, when I look at the it-awx-tls-secret.yml file the tls.crt and tls.secret values do not match the values in the .pem and .key file.

After I generate a new certificate, how do I populate the respective .crt and .key in the awx-tls-secret.yml file?

Thanks for all your help!

Below are files I found in the base directory from the previous maintainer:

-rw-r--r--  1 1889601905 1889601905  1890 May 17  2022 it-awx-ssl.cer
-rw-rw-r--  1 1889601905 1889601905  1700 May 17  2022 it-awx-ssl.csr
-rw-------  1 1889601905 1889601905  3414 May 17  2022 it-awx-ssl.key
-rw-------  1 1889601905 1889601905  1675 May 17  2022 it-awx-ssl.pem
-rw-------  1 1889601905 1889601905  3243 May 17  2022 it-awx-ssl_decr.key
-rw-rw-r--  1 1889601905 1889601905  6972 May 17  2022 it-awx-tls-secret.yml
-rw-rw-r--  1 1889601905 1889601905  6972 Aug 31  2022 it-awx-tls.yml

It’s difficult to know what the previous maintainer did, and to be honest it looks to me as though they tried all manner of things … :wink:

If you have the new certificate, which you might have created from an existing CSR, then you’ll probably not have to replace the .key as it remains identical.

The certificate will probably be a PEM encoded file (-- BEGIN CERTIFICATE --) which goes in the .pem file. Before overwriting that, you can compare its content with the .cer file (possibly DER-encoded on Windows?)

If file it*.cer says something like binary or data, then:

$ openssl x509 -in it-awx-ssl.cer -inform der -noout -subject

will show you the certificate subject. If it’s not a binary/data file, omit -inform der (or replace by -inform pem). You can compare with the *.pem file by replacing the -in file in above command.

Thank you so much for your suggestions. Yes, it’s hard to get into someone’s mind - especially when they are no longer around. There wasn’t a design spec or document containing a how-to for updating the SSL cert.

Moving forward, I’ll take an offline snapshot of the VM - just in case things go sideways I can easily rollback.

What does the apply command do exactly?

I understand it applies the config in kustomize.config. But the first resource in the config file is a repo (see below from my cat output). Does it pull and install the latest operator version?

kind: Kustomization
  # Find the latest tag here:
  - it-awx.yml

# Set the image tags to match the git version from above
  - name:
    newTag: 0.21.0

# Specify a custom namespace in which to install AWX
namespace: awx

I get the following error now when syncing inventory

May 17 00:15:48 hq-it-awx k3s[3864180]: I0517 00:15:48.422518 3864180 kubelet_pods.go:891] "Unable to retrieve pull secret, the image pull may not succeed." pod="awx/awx-operator-controller-manager-675865446d-cs7vf" secret="" err="secret \"redhat-operators-pull-secret\" not found"