Issue with Custom CA Certs

I have installed AWX operator on minikube following the guide here.

I am fairly new to minikube/k8s so please forgive my ignorance.

My system is in an offline environment so I have had to take the steps below to set it up:

  1. Install dependencies sudo dnf install minikube.rpm gcc unzip git
  2. Load the k8s images with minikube image load
  3. Copy the git repo to the machine and add binaries to bin/
  4. Checkout the relevant tags git checkout tags/2.9.0; export VERSION=2.9.0
  5. Compile make deploy
  6. Create kustomization.yaml with the contents
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - config/default/
images:
  - name: quay.io/ansible/awx-operator
    newTag: 2.9.0
namespace: awx
  1. Apply and wait for the operator pods kubectl apply -k .
  2. Create an awx-demo.yml file with the following info:
---
apiVersion: awx.ansible.com/v1beta1
kind: AWX
metadata:
  name: awx-demo
spec:
  service_type: nodeport
  extra_settings:
    - setting: CSRF_TRUSTED_ORIGINS
      value:
        - https://<hostname>
        - https://<fqdn>
  1. Add the new demo file to kustomization.yaml
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - config/default/
  - awx-demo.yml
images:
  - name: quay.io/ansible/awx-operator
    newTag: 2.9.0
namespace: awx
  1. Apply and wait for the task and web pods kubectl apply -k .

This is a high-level explanation of the steps I took. There is an nginx proxy sitting in front of the setup which allows web access, which works as expected.

The issue I am having is following the custom CA cert guide here.

When I create the secrets with the following commands:

kubectl create secret generic awx-demo-custom-certs --from-file=ldap-ca.crt=RootCA.pem --from-file=bundle-ca.crt=ca-bundle.crt

The secrets are created and I can inspect them with no issues kubectl get secret awx-demo-custom-certs -o jsonpath=".data.ldap-ca\.crt" | base64 --decode
The issue comes when I add the certs to the awx-demo.yml file

---
apiVersion: awx.ansible.com/v1beta1
kind: AWX
metadata:
  name: awx-demo
spec:
  service_type: nodeport
  ldap_cacert_secret: awx-demo-custom-certs
  bundle_cacert_secret: awx-demo-custom-certs
  extra_settings:
    - setting: CSRF_TRUSTED_ORIGINS
      value:
        - https://<hostname>
        - https://<fqdn>

When added like this the pods will attempt to start up but fail, endlessly cycling. I’ve tried to get the logs for this but the only thing I see is that the container is no longer considered valid.

I did find some other guides that walked through this operation on k3s but nothing seems to work.
I’ve also tried creating the certificates with the kustomization.yaml file which worked but I got the same error when trying to deploy.

Any help with this would be greatly appreciated.

I would try it with the latest awx-operator version 2.19.1. With this version custom CAs are working for me.
I’ve fixed some bugs regarding custom CAs in operator in older versions.

I’ll import this version and give this a try.

I will let you know the outcome

Yea it’s dropped straight in and I can see the certs inside the pods now.