Node or Pod IP based on destination

[knowosielski-SJSU (Keith)]

This one is strange. I’ve just migrated my AWX instance from an on-prem k8s cluster to a GKE cluster in GCP. Everything is working as expected with some minor changes such as the source IP needed to be from the pod subnet instead of the node IP’s. While testing and setting up connectivity it was discovered that certain destination IP subnets are causing the source IP to be one of the Node IP’s (same as before when we were on-prem).

Any insight as to why this is happening?
Is there a way to change this behavior?
Is this something with awx or is this a kuberntes GKE thing?

AWX Version: 23.5.0
AWX Operator: 2.28.0

[kurokobo]

Hi, I don’t know much about your actual situation, but the AWX does nothing over the source IP addresses of the traffics AWX makes.
Therefore, I believe your issue is due to the GKE-thing instead of something with AWX.

I have not configured this myself, so I cannot give specific advice, but if you want to control source IP address, perhaps there is a clue around Cloud NAT or Egress NAT Policy on GKE.

• Cloud NAT overview  |  Google Cloud
• Public NAT  |  Google Cloud
• Utiliser Public NAT avec GKE  |  Google Cloud
• Use Egress NAT Policy to configure IP masquerade in Autopilot clusters  |  Google Kubernetes Engine (GKE)  |  Google Cloud

[knowosielski-SJSU (Keith)]

Thanks for your reply. I also don’t believe the issue is within AWX, however I’ve never encountered this before. I’m still fairly new to k8s and gke.

The GKE cluster is a standard cluster running Ubuntu Container OS. Its a private cluster and there’s no nat’s or external public addresses.

[knowosielski-SJSU (Keith)]

Update on this in case anyone else has this issue.

GKE/GCP treated the addresses in question as public addresses and therefore tried to route them accordingly. We decied to address the issue by re-IPing the devices because they shouldn’t have been on public addresses to begin with.