AWX behind Caddy Reverse Proxy

Get Help
1

Hello Guys,

I am currently looking into running an AWX Instance behind a reverse proxy. The issue is, that we can not get the TLS connection to the server set up and the remote hop node wont connect to the AWX.

We are running a caddy as a reverse proxy. Due to internal policies, we can not let the hop node connect directly to the AWX.

Maybe a TLS Passthrough would solve the issue?

I am currently at my wits end and am looking for some help here.

Addidtional information:

  • AWX Version 24.6.1
  • AWX Operator 2.9.1
  • running on k3s
  • Caddy Version 2.11.2 running as a Docker Container with a Layer4 Plugin
2

The issue is almost certainly Receptor mTLS colliding with Caddy’s TLS handling. AWX uses Receptor for its execution mesh on port 27199 by default, and the certificates Receptor uses are issued by AWX’s own internal CA, not something Caddy can validate or terminate cleanly. For the Caddy Layer4 plugin, you want TCP passthrough for port 27199, not TLS proxy mode. The distinction matters: TLS proxy has Caddy terminate and re-encrypt, in fairness which breaks Receptor’s mutual certificate checks entirely. Passthrough just forwards raw TCP bytes and lets Receptor handle its own handshake end-to-end. Layer4 Caddyfile syntax for this:

{
  layer4 {
    0.0.0.0:27199 {
      route {
        proxy {
          upstream awx-control-plane:27199
        }
      }
    }
  }
}

No tls block inside that route, the moment you add tls, Caddy tries to participate in the handshake and Receptor rejects it due to certificate mismatch. Second thing to check: on the hop node’s receptor.conf, the peer address must be the external Caddy hostname, not the in-cluster service DNS. If it’s still pointing at something like awx-service.awx.svc.cluster.local:27199, the hop node connects directly and bypasses Caddy, fine internally but breaks external execution nodes completely. Third: look at Administration → Instance Groups in AWX. The Control node and Hop node should show correct peer relationships. If the hop node shows “not connected” or “unavailable” there, the Receptor mesh never established regardless of Caddy config. One k3s-specific gotcha: if you’re using NodePort or LoadBalancer service for port 27199, confirm the service actually exposes that port externally. MetalLB and k3s ServiceLB behave differently here, and it’s easy to have a service that looks configured but the port isn’t reachable outside the cluster.