Error In kustomization.yml - Unknown field spec.containers

[Dustin]

I’m still very new to Kubernetes, and I can’t get an environment var passed to the AWX-EE container. I got my AWX running with a lot of trial, error, and reboots. As far as I know, I’m running k3s rancher in single node just for this AWX install.

Thank you @kurokobo for your GitHub page. That got me 99% of the way there.

Now, I’m trying to set REQUESTS_CA_BUNDLE in my awx-ee container to point to the RHEL ca-bundle.crt so that ansible-galaxy installations trust my proxy cert and I can install collections to my environment.

Here’s my kustomization.yaml file

---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  # Find the latest tag here: https://github.com/ansible/awx-operator/releases
  - github.com/ansible/awx-operator/config/default?ref=2.9.0
  - awx.yaml

# Set the image tags to match the git version from above
images:
  - name: quay.io/ansible/awx-operator
    newTag: 2.9.0

# Specify a custom namespace in which to install AWX
namespace: awx

And my awx.yaml

---
apiVersion: awx.ansible.com/v1beta1
kind: AWX
metadata:
  name: awx
spec:
  containers:
    - name: awx-ee
      env:
        - name: REQUESTS_CA_BUNDLE
          value: /path/to/ca-bundle.crt

  admin_user: admin
  admin_password_secret: awx-admin-password

  ingress_type: ingress
  ingress_tls_secret: awx-tls
  hostname: awxlab.mydomain.com

  extra_settings:
  - setting: CSRF_TRUSTED_ORIGINS
    value:
      - https://awxlab.mydomain.com

The error I’m getting
: strict decoding error: unknown field "spec.containers"

There’s a debug included in the error message as well, but I haven’t had a chance to scrub it for sensitive data yet. I did look through the debug and it pretty much just looks like the awx.yaml file converted to json.

[kurokobo]

Hi, thank you for using my guide and your shout-out

@Dustin

spec:
  containers:
    - name: awx-ee
      env:
        - name: REQUESTS_CA_BUNDLE
          value: /path/to/ca-bundle.crt

containers under spec for kind: AWX is not allowed, since no such field is defined in CRD: awx-operator/config/crd/bases/awx.ansible.com_awxs.yaml at devel · ansible/awx-operator · GitHub

Try bundle_cacert_secret and ee_extra_env instead.

• Trusting a custom certificate authority - Ansible AWX Operator Documentation
• Exporting environment variables to containers - Ansible AWX Operator Documentation

[Denney-tech (Caleb Denney)]

Thanks @kurokobo! That’s the better way to do this.

I recognize @Dustin’s snippet from my reply the other day. The way I was thinking to was to customize the pod spec in the AWX UI > Administration > Instance Groups > edit default (Container Group, and any other CG) and click the checkbox to customize pod specifications. Then my snippet can be merged into the custom pod spec. REQUESTS_CA_BUNDLE is definitely better suited in the ee_extra_env so it applies globally, but knowing how to do this in the UI can be useful if you just need something defined in a specific container group (like setting ANSIBLE_JINJA2_NATIVE for e.g.)

[kurokobo]

Ah thanks for providing the background.

Customizing pod spec by defining new Container Group is powerful way, but it can’t affect project updating, since project sync job is processed on the awx-ee container in task pod itself, not on the automation job pod.

I assume that @Dustin requires trusting custom CA certs for ansible-galaxy that will be invoked during project updates by placing collections/requirements.yml on project root, so I think Container Group cannot help.

An applicable solution is creating new secret and adding bundle_cacert_secret to AWX CR. I guess adding REQUESTS_CA_BUNDLE is not required for ansible-galaxy.

My guide also has this topic: awx-on-k3s/tips/trust-custom-ca.md at main · kurokobo/awx-on-k3s · GitHub

[Dustin]

I’ve seen your replies and intend on trying them out. Just working around other priorities now. I’ll mark you reply as solution once I get it tested and working.

Thanks again!

[Dustin]

I see where I messed up. When I first was going through those instructions and documentation, I was brand new to Kubernetes and AWX-Operator. I generated the secret in the kustomization and then also created it manually from the CLI; which was causing a conflict.

Thanks again, @kurokobo and @Denney-tech! Unfortunately, I can only mark one as solution and Kurokobo’s reply was more thorough and had documentation links.

[Denney-tech (Caleb Denney)]

No worries, @kurokobo was right. Adding REQUESTS_CA_BUNDLE to the container groups won’t help where ansible-galaxy is running, but also probably isn’t technically needed since all you should need to do is add the bundle secret (which you noted were you were accidentally creating a conflict yourself). I think it’s still be useful to add the environment variable though, as I have found occasionally that there are some modules/plugins that just won’t use the updated system ca-bundle unless you specifically tell them to with that env var.