Firewall interfering on AWX

Get Help
, ,
1

Hi, there!

I’ve just installed Fedora on my notebook. I’ve also installed a vanilla Kubernetes on it and AWX.

AWX seems to be working correctly (could login and update admin’s password). The AWX pods run smoothly, except when the firewall is running: the awx-web pod starts to crash. (EDIT: turning off the firewall and turning on again allows the awx-web pod to keep running without restarts, but web GUI reports HTTP 500 (server error); turning off the firewall instantly put the GUI to work again).

The ports open on my notebook:

internal (default, active)
  target: default
  ingress-priority: 0
  egress-priority: 0
  icmp-block-inversion: no
  interfaces: wlp3s0
  sources: 
  services: cockpit dhcpv6-client http https mdns samba-client ssh
  ports: 6443/tcp 2379-2380/tcp 10250/tcp 10256/tcp 10259/tcp 10257/tcp 30000-32767/tcp 30000-32767/udp 27199/tcp
  protocols: 
  forward: yes
  masquerade: no
  forward-ports: 
  source-ports: 
  icmp-blocks: 
  rich rules:

So every K8s port is mentioned, as well ports used by AWX: 80, 443 and 27199.

What am I missing?

2

This is likely neither an AWX or K8s issue.

Firewalld is known to rebuild iptables and break the rules your container runtime sets up.

This can get more complex dependening on what CNI you’re using. The issue will be at the intersection of the runtime&CNI, for example with Rancher: Known Issues and Limitations | RKE2

If you’d like to go down the rabbithole, you can compare the state of iptables -S with firewalld disabled (so disable it → reboot → get things happy → iptables -S). Then you can break it and compare the state.

That is a fun exercise to understand “what gives”. Unless you’ve got days&days of time to fix it, I’d look for a way to get firewalld as far away from your cluster as possible :smiley:

I’d wager podman’s networking remains sane on Fedora - so maybe switching to a single-node kind cluster would do the trick. (https://kind.sigs.k8s.io/)

podman support on it is experimental, but last I checked, I managed to run a simple deployment on it.