AWX-operator redis image backoff error - where are creds for docker.io?

pdin
Wed, 03 Dec 2025 15:08:08 +0000
Get Help

I have an AWX operator running Ansible in Kubernetes. The redis pods is throwing the image backoff error:
Events:
Type Reason Age From Message

Warning Failed 8m58s (x5330 over 10d) kubelet (combined from similar events): Error: ImagePullBackOff
Normal BackOff 3m56s (x73591 over 10d) kubelet Back-off pulling image “docker.io/redis:7”
Warning Failed 3m56s (x72392 over 10d) kubelet Error: ImagePullBackOff

I think this is because there is not docker.io secrets in this install. How can I setup creds separately? I do not think the ee creds are the same needed for redis from what I can see.

Here's the full error log:

cat /tmp/troubleshooting.txt
Name:             awx-demo-web-5bf48f486-bgpc2
Namespace:        awx
Priority:         0
Service Account:  awx-demo
Node:             usll-dev-smi-zone-a-869987d56f-vznnr/100.112.138.10
Start Time:       Fri, 21 Nov 2025 17:38:10 -0600
Labels:           app.kubernetes.io/component=awx
                  app.kubernetes.io/managed-by=awx-operator
                  app.kubernetes.io/name=awx-demo-web
                  app.kubernetes.io/operator-version=2.19.1
                  app.kubernetes.io/part-of=awx-demo
                  app.kubernetes.io/version=24.6.1
                  pod-template-hash=5bf48f486
Annotations:      checksum-configmaps-config: e234f6893a95e0adc322f334558e86ff383b24ab
                  checksum-secret-bundle_cacert: da39a3ee5e6b4b0d3255bfef95601890afd80709
                  checksum-secret-ldap_cacert: da39a3ee5e6b4b0d3255bfef95601890afd80709
                  checksum-secret-receptor_ca: 3754ffa7d25781fee7a907d25f2711d34a23f250
                  checksum-secret-receptor_work_signing: 81f7f9630bd1437a5628487c5852f278d1576ede
                  checksum-secret-route_tls: da39a3ee5e6b4b0d3255bfef95601890afd80709
                  checksum-secret-secret_key: c87c7583c49b4f99207039b312db5d212592bb87
                  checksum-secrets-app_credentials: c1390276d67609884acac8fd5be2e2f7b38f9e55
                  checksum-storage-persistent: adc83b19e793491b1c6ea0fd8b46cd9f32e592fc
                  kubectl.kubernetes.io/default-container: awx-demo-web
                  kubectl.kubernetes.io/restartedAt: 2025-09-08T11:56:05-05:00
Status:           Pending
IP:               192.168.3.82
IPs:
  IP:           192.168.3.82
Controlled By:  ReplicaSet/awx-demo-web-5bf48f486
Containers:
  redis:
    Container ID:
    Image:         docker.io/redis:7
    Image ID:
    Port:          <none>
    Host Port:     <none>
    Args:
      redis-server
      /etc/redis.conf
    State:          Waiting
      Reason:       ImagePullBackOff
    Ready:          False
    Restart Count:  0
    Requests:
      cpu:        50m
      memory:     64Mi
    Environment:  <none>
    Mounts:
      /data from awx-demo-redis-data (rw)
      /etc/redis.conf from awx-demo-redis-config (ro,path="redis.conf")
      /var/run/redis from awx-demo-redis-socket (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-8zbx2 (ro)
  awx-demo-web:
    Container ID:  containerd://5028dfb71ac7bbaebcb46ff59e063e95ef085b873aa82f9e7dbb9fe661fe517e
    Image:         quay.io/ansible/awx:24.6.1
    Image ID:      quay.io/ansible/awx@sha256:5d47e132014ae4c6a026a2e8f19ce7b3cf3c8f313d59e44fdfa95cd9f3669209
    Ports:         8052/TCP, 443/TCP
    Host Ports:    0/TCP, 0/TCP
    Args:
      /usr/bin/launch_awx_web.sh
    State:          Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       Completed
      Exit Code:    0
      Started:      Tue, 02 Dec 2025 15:52:53 -0600
      Finished:     Tue, 02 Dec 2025 15:53:25 -0600
    Ready:          False
    Restart Count:  2793
    Requests:
      cpu:     100m
      memory:  128Mi
    Environment:
      AWX_COMPONENT:           web
      SUPERVISOR_CONFIG_PATH:  /etc/supervisord_web.conf
      MY_POD_NAMESPACE:        awx (v1:metadata.namespace)
      MY_POD_IP:                (v1:status.podIP)
      UWSGI_MOUNT_PATH:        /
    Mounts:
      /etc/nginx/nginx.conf from awx-demo-nginx-conf (ro,path="nginx.conf")
      /etc/receptor/tls/ca/mesh-CA.crt from awx-demo-receptor-ca (ro,path="tls.crt")
      /etc/receptor/tls/ca/mesh-CA.key from awx-demo-receptor-ca (ro,path="tls.key")
      /etc/receptor/work_public_key.pem from awx-demo-receptor-work-signing (ro,path="work-public-key.pem")
      /etc/tls from awx-tls-volume (ro)
      /etc/tower/SECRET_KEY from awx-demo-secret-key (ro,path="SECRET_KEY")
      /etc/tower/conf.d/credentials.py from awx-demo-application-credentials (ro,path="credentials.py")
      /etc/tower/conf.d/execution_environments.py from awx-demo-application-credentials (ro,path="execution_environments.py")
      /etc/tower/conf.d/ldap.py from awx-demo-application-credentials (ro,path="ldap.py")
      /etc/tower/settings.py from awx-demo-settings (ro,path="settings.py")
      /etc/tower/uwsgi.ini from awx-demo-uwsgi-config (ro,path="uwsgi.conf")
      /var/run/awx-rsyslog from rsyslog-socket (rw)
      /var/run/redis from awx-demo-redis-socket (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-8zbx2 (ro)
  awx-demo-rsyslog:
    Container ID:  containerd://5a29204189e748e21ed2aacdc1f02ecdfb79730f6cdf4e26f9b951839895f8ed
    Image:         quay.io/ansible/awx:24.6.1
    Image ID:      quay.io/ansible/awx@sha256:5d47e132014ae4c6a026a2e8f19ce7b3cf3c8f313d59e44fdfa95cd9f3669209
    Port:          <none>
    Host Port:     <none>
    Args:
      /usr/bin/launch_awx_rsyslog.sh
    State:          Running
      Started:      Fri, 21 Nov 2025 17:38:13 -0600
    Ready:          True
    Restart Count:  0
    Requests:
      cpu:     100m
      memory:  128Mi
    Environment:
      SUPERVISOR_CONFIG_PATH:  /etc/supervisord_rsyslog.conf
    Mounts:
      /etc/tower/SECRET_KEY from awx-demo-secret-key (ro,path="SECRET_KEY")
      /etc/tower/conf.d/credentials.py from awx-demo-application-credentials (ro,path="credentials.py")
      /etc/tower/settings.py from awx-demo-settings (ro,path="settings.py")
      /var/run/awx-rsyslog from rsyslog-socket (rw)
      /var/run/redis from awx-demo-redis-socket (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-8zbx2 (ro)
Conditions:
  Type                        Status
  PodReadyToStartContainers   True
  Initialized                 True
  Ready                       False
  ContainersReady             False
  PodScheduled                True
Volumes:
  awx-demo-receptor-ca:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  awx-demo-receptor-ca
    Optional:    false
  awx-demo-receptor-work-signing:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  awx-demo-receptor-work-signing
    Optional:    false
  awx-demo-application-credentials:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  awx-demo-app-credentials
    Optional:    false
  awx-demo-secret-key:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  awx-demo-secret-key
    Optional:    false
  awx-demo-settings:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      awx-demo-awx-configmap
    Optional:  false
  awx-demo-nginx-conf:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      awx-demo-awx-configmap
    Optional:  false
  awx-demo-redis-config:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      awx-demo-awx-configmap
    Optional:  false
  awx-demo-uwsgi-config:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      awx-demo-awx-configmap
    Optional:  false
  awx-demo-redis-socket:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:
    SizeLimit:  <unset>
  awx-demo-redis-data:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:
    SizeLimit:  <unset>
  rsyslog-socket:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:
    SizeLimit:  <unset>
  receptor-socket:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:
    SizeLimit:  <unset>
  awx-demo-receptor-config:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      awx-demo-awx-configmap
    Optional:  false
  awx-tls-volume:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  awx-tls
    Optional:    false
  kube-api-access-8zbx2:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   Burstable
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/memory-pressure:NoSchedule op=Exists
                             node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason   Age                      From     Message
  ----     ------   ----                     ----     -------
  Warning  Failed   8m58s (x5330 over 10d)   kubelet  (combined from similar events): Error: ImagePullBackOff
  Normal   BackOff  3m56s (x73591 over 10d)  kubelet  Back-off pulling image "docker.io/redis:7"
  Warning  Failed   3m56s (x72392 over 10d)  kubelet  Error: ImagePullBackOff
[ephidin@uslllx0006 ~]$

1 post - 1 participant