Ability to allow inbound connection to AWX receptor mesh on Kubernetes

[TheRealHaoLiu (Hao Liu)]

@kurokobo @tanganellilore we got a good bits of the work done in
feature_awx-mesh-ingress branch on awx and awx-operator

right now if you try to use it it should work

for the awx-operator change we mostly focused on OCP with Route as the ingress

If anyone is interested would u like to contribute the bits needed to get this to work well on non OCP K8S?

[kurokobo]

Thank you for informing me, I’ll give it a little try in a few days.

[kurokobo]

[AWXMeshIngress] The port for internal receptor address should be 27199 instead of 443

[AWXMeshIngress] "Something went wrong" in Peers tab for Instances

[AWXMeshIngress] Adding new instance that has AWXMeshIngress node as peer causes 400 error

[kurokobo]

@TheRealHaoLiu @fosterseth
Is it feasible for AWX Operator to support deployment of IngressRouteTCP for Traefik in addition to Route and Ingress?
I can send PR for this, but It would also be possible to deploy AWXMeshIngress CR with ingress_type: none and have users create their own IngressRouteTCP (In this case AWX Operator does not support IngressRouteTCP).

Traefik (default ingress controller for k3s) requires a TCP Router for TLS passthrough, which consists of a CR called IngressRouteTCP rather than the usual Ingress resource.

Technically, I already have an implementation in my environment that allows jobs to run on Executon Node via Internal Hop Node using IngressRouteTCP.

Draft PR: wip: add ingress and ingressroutetcp for awxmeshingress by kurokobo · Pull Request #1646 · ansible/awx-operator · GitHub

[TheRealHaoLiu (Hao Liu)]

AWX Community Meeting 2024-02-13

Thanks everyone! the feature landed in AWX now!

check it out!

AWX: 23.8.0
AWX-Operator: 2.12.0

[yoonmin1030 (yoonmin)]

The remote EE node from outside of k8s cluster, how to reach the hop node in the k8s cluster with port 443 ? does the loadbalancer in k8s cluster tell ?
I have installed bundle to remote EE node, but running health check tells the remote EE node is not in the receptor mesh.

what i understand is

1. in the k8s cluster, awxmeshingress takes pods to service.
2. outside of k8s cluster, ingress (ingress-nginx) takes the remote EE node to service.

please take a look what I am missing.

this is my service for awxmeshingress

apiVersion: v1
kind: Service
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: ‘{“apiVersion”:“v1”,“kind”:“Service”,“metadata”:{“name”:“mesh-ingress-1”,“namespace”:“awx”},“spec”:{“ports”:[{“name”:“ws”,“port”:27199,“targetPort”:27199}],“selector”:{“app.kubernetes.io/name":“mesh-ingress-1”},“type”:"ClusterIP”}}’
creationTimestamp: “2024-04-12T07:43:01Z”
name: mesh-ingress-1
namespace: awx
ownerReferences:

• apiVersion: awx.ansible.com/v1alpha1
  kind: AWXMeshIngress
  name: mesh-ingress-1
  uid: xxx
  resourceVersion: “21687088”
  uid: xxx
  spec:
  clusterIP: 10.x.x.x
  clusterIPs:
• 10.x.x.x
  internalTrafficPolicy: Cluster
  ipFamilies:
• IPv4
  ipFamilyPolicy: SingleStack
  ports:
• name: ws
  port: 27199
  protocol: TCP
  targetPort: 27199
  selector:
  app.kubernetes.io/name: mesh-ingress-1
  sessionAffinity: None
  type: ClusterIP
  status:
  loadBalancer: {}

===============================================
following is service for ingress-nginx-controller

apiVersion: v1
kind: Service
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{“apiVersion”:“v1”,“kind”:“Service”,“metadata”:{“annotations”:{},“labels”:{“app.kubernetes.io/component":“controller”,“app.kubernetes.io/instance”:“ingress-nginx”,“app.kubernetes.io/name”:“ingress-nginx”,“app.kubernetes.io/part-of”:“ingress-nginx”,“app.kubernetes.io/version”:“1.10.0”},“name”:“ingress-nginx-controller”,“namespace”:“ingress-nginx”},“spec”:{“externalTrafficPolicy”:“Local”,“ipFamilies”:[“IPv4”],“ipFamilyPolicy”:“SingleStack”,“ports”:[{“appProtocol”:“http”,“name”:“http”,“port”:80,“protocol”:“TCP”,“targetPort”:“http”},{“appProtocol”:“https”,“name”:“https”,“port”:443,“protocol”:“TCP”,“targetPort”:“https”}],“selector”:{“app.kubernetes.io/component”:“controller”,“app.kubernetes.io/instance”:“ingress-nginx”,“app.kubernetes.io/name”:“ingress-nginx”},“type”:"LoadBalancer”}}
creationTimestamp: “2024-04-03T09:20:23Z”
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.10.0
name: ingress-nginx-controller
namespace: ingress-nginx
resourceVersion: “18611647”
uid: xxx
spec:
allocateLoadBalancerNodePorts: true
clusterIP: 10.x.x.x
clusterIPs:

• 10.x.x.x
  externalTrafficPolicy: Local
  healthCheckNodePort: 31209
  internalTrafficPolicy: Cluster
  ipFamilies:
• IPv4
  ipFamilyPolicy: SingleStack
  ports:
• appProtocol: http
  name: http
  nodePort: 31789
  port: 80
  protocol: TCP
  targetPort: http
• appProtocol: https
  name: https
  nodePort: 32148
  port: 443
  protocol: TCP
  targetPort: https
  selector:
  app.kubernetes.io/component: controller
  app.kubernetes.io/instance: ingress-nginx
  app.kubernetes.io/name: ingress-nginx
  sessionAffinity: None
  type: LoadBalancer
  status:
  loadBalancer: {}

[kurokobo]

@yoonmin1030
Hi, could you please create a new topic on Get Help category?

[yoonmin1030 (yoonmin)]

I made it as a topic.

Question about adding remote EE node to AWX k8s cluster

The remote EE node from outside of k8s cluster, how to reach the hop node in the k8s cluster with port 443 ? does the loadbalancer in k8s cluster tell ? I have installed bundle to remote EE node, but running health check tells the remote EE node is not in the receptor mesh. what i understand is in the k8s cluster, awxmeshingress takes pods to service. outside of k8s cluster, ingress (ingress-nginx) takes the remote EE node to service. please take a look what I am missing. this is my servic…

please help me to get more understanding of the topic “awxmeshingress and receptor”