# Running two versions of awx in the same K8s cluster? **URL:** \ **Category:** AWX Project\ **Tags:** awx, kubernetes\ **Created:** [July 29, 2021, 4:26pm UTC](https://forum.ansible.com/t/running-two-versions-of-awx-in-the-same-k8s-cluster/35566 "2021-07-29T16:26:06Z")\ **Posts on this page:** 19\ **Page:** 1
**Author:** ![howardjones](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/howardjones/32/4169_2.png) [@howardjones](https://forum.ansible.com/u/howardjones)\ **Post date:** [July 29, 2021, 4:26pm UTC](https://forum.ansible.com/t/running-two-versions-of-awx-in-the-same-k8s-cluster/35566/1 "2021-07-29T16:26:06Z")
We have awx 15.0.1 running, installed via AWX Operator 0.6.0. I’d like to run up an awx 19.2.2 instance to test before moving over. Is it possible to do this on the same cluster? Does the 0.12.0 operator understand the old AWXS CRD as well as the new one? Or can you run two Operators that will each deal with only their own version? (I think no, since the API version number is the same in both AWXS CRDs) Does anyone else do this? I just want to check out EEs (and have others look too, so minikube doesn’t really help) before plunging into using 19.2.2 ---
**Author:** ![tin](https://avatars.discourse-cdn.com/v4/letter/t/5f9b8f/32.png) [@tin](https://forum.ansible.com/u/tin)\ **Post date:** [July 30, 2021, 5:57am UTC](https://forum.ansible.com/t/running-two-versions-of-awx-in-the-same-k8s-cluster/35566/2 "2021-07-30T05:57:31Z")
Yes, I do that. Just create another namespace and edit your yaml file for the Operator. Note that I am using AWX Operator 0.12.0. I have several different versions of AWX in my GKE cluster. Using different namespaces for each one. Tin ---
**Author:** ![Cnu\_k](https://avatars.discourse-cdn.com/v4/letter/c/a88e4f/32.png) [@Cnu\_k](https://forum.ansible.com/u/Cnu_k)\ **Post date:** [September 15, 2021, 1:58pm UTC](https://forum.ansible.com/t/running-two-versions-of-awx-in-the-same-k8s-cluster/35566/3 "2021-09-15T13:58:53Z")
Hello Tin, Would you please share changes needed to limit the operator install to a custom namespace, including the operator, CRDs, and AWX instance? Based on what I read, it appears that we can use a custom namespace for creating the instance, but not sure what needs to be done for the operator install itself. Thanks ---
**Author:** ![Wei-Yen\_Tan](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/wei-yen_tan/32/4111_2.png) [@Wei-Yen\_Tan](https://forum.ansible.com/u/Wei-Yen_Tan)\ **Post date:** [September 15, 2021, 2:08pm UTC](https://forum.ansible.com/t/running-two-versions-of-awx-in-the-same-k8s-cluster/35566/4 "2021-09-15T14:08:00Z")
The kustomize.yaml does it. [https://github.com/kurokobo/awx-on-k3s/blob/main/base/kustomization.yaml](https://github.com/kurokobo/awx-on-k3s/blob/main/base/kustomization.yaml) See? Look at namespace and see how it is including the operator yaml. ---
**Author:** ![Cnu\_k](https://avatars.discourse-cdn.com/v4/letter/c/a88e4f/32.png) [@Cnu\_k](https://forum.ansible.com/u/Cnu_k)\ **Post date:** [September 15, 2021, 3:39pm UTC](https://forum.ansible.com/t/running-two-versions-of-awx-in-the-same-k8s-cluster/35566/5 "2021-09-15T15:39:50Z")
Thanks again. If my understanding is correct, the operator is still a separate install, it’s AWX instance and other resources that are namespace bound. Would it be possible to limit the operator as well to a namespace? ---
**Author:** ![Wei-Yen\_Tan](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/wei-yen_tan/32/4111_2.png) [@Wei-Yen\_Tan](https://forum.ansible.com/u/Wei-Yen_Tan)\ **Post date:** [September 15, 2021, 3:42pm UTC](https://forum.ansible.com/t/running-two-versions-of-awx-in-the-same-k8s-cluster/35566/6 "2021-09-15T15:42:17Z")
Have a look at the kustomize.yaml and look at the name space. nes (30 sloc) 610 Bytes ---
**Author:** ![Cnu\_k](https://avatars.discourse-cdn.com/v4/letter/c/a88e4f/32.png) [@Cnu\_k](https://forum.ansible.com/u/Cnu_k)\ **Post date:** [September 15, 2021, 3:48pm UTC](https://forum.ansible.com/t/running-two-versions-of-awx-in-the-same-k8s-cluster/35566/7 "2021-09-15T15:48:20Z")
Sorry for not making it clear. I see that AWX instances can be created in difference namespace, I was referring to limiting the operator install itself to a single namespace(not cluster-scoped). I see that [https://github.com/ansible/awx-operator/blob/0.12.0/deploy/awx-operator.yaml](https://github.com/ansible/awx-operator/blob/0.12.0/deploy/awx-operator.yaml) has few hard-coded entries with ‘namespace: default’, I will change these in the local copy and see if that would work since all other CRDs are supposed to work in ’ scope: Namespaced’. Thanks ---
**Author:** ![Wei-Yen\_Tan](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/wei-yen_tan/32/4111_2.png) [@Wei-Yen\_Tan](https://forum.ansible.com/u/Wei-Yen_Tan)\ **Post date:** [September 15, 2021, 8:04pm UTC](https://forum.ansible.com/t/running-two-versions-of-awx-in-the-same-k8s-cluster/35566/8 "2021-09-15T20:04:46Z")
That’s actually a kubectl thing then. Not specific to awx. Look at this example ```auto kubectl apply -f pod.yaml --namespace=test ``` ---
**Author:** ![howardjones](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/howardjones/32/4169_2.png) [@howardjones](https://forum.ansible.com/u/howardjones)\ **Post date:** [September 15, 2021, 8:24pm UTC](https://forum.ansible.com/t/running-two-versions-of-awx-in-the-same-k8s-cluster/35566/9 "2021-09-15T20:24:22Z")
But does that limit the Operator to only looking for CRDs in its own namespace? That's not how they normally work. For e.g. nginx ingress, you have to start defining special classes to filter which CRDs it is interested in. It'd be really useful to be able to test the next Operator+awx version in a different namespace of the same cluster... ---
**Author:** ![tin](https://avatars.discourse-cdn.com/v4/letter/t/5f9b8f/32.png) [@tin](https://forum.ansible.com/u/tin)\ **Post date:** [September 16, 2021, 1:00am UTC](https://forum.ansible.com/t/running-two-versions-of-awx-in-the-same-k8s-cluster/35566/10 "2021-09-16T01:00:03Z")
I used the same operator, version 0.12.0, for the different versions of AWX in the different namespaces. AFAIK, operator 0.12.0 works fine for any version of AWX from 19.0 up. I built my own AWX image and pushed it to our own image repo due to security reasons. Basically, you specify the namespace in the ‘metadata’ section of your yml deploy file. Here is a snippet of mine: ---
**Author:** ![Cnu\_k](https://avatars.discourse-cdn.com/v4/letter/c/a88e4f/32.png) [@Cnu\_k](https://forum.ansible.com/u/Cnu_k)\ **Post date:** [September 16, 2021, 10:28pm UTC](https://forum.ansible.com/t/running-two-versions-of-awx-in-the-same-k8s-cluster/35566/11 "2021-09-16T22:28:22Z")
I have installed the operator and AWX instance into the namespace, had to update serviceaccount and clusterrolebinding YAML entries to change from ‘default’ to a custom namespace. I now see the operator CR, CRDs, pods and other resources in the same namespace as the instance where it’s configured to run. I believe this would help to keep each AWX set up separate from other. BTW, we also have a need to customize [quay.io/ansible/awx:19.2.2](http://quay.io/ansible/awx:19.2.2) image to add our own tools and push to an internal registry, where can we see Dockerfile for these images to get an idea on included tools and utilities? Thanks ---
**Author:** ![tin](https://avatars.discourse-cdn.com/v4/letter/t/5f9b8f/32.png) [@tin](https://forum.ansible.com/u/tin)\ **Post date:** [September 17, 2021, 4:12am UTC](https://forum.ansible.com/t/running-two-versions-of-awx-in-the-same-k8s-cluster/35566/12 "2021-09-17T04:12:12Z")
The instructions are here: [https://github.com/ansible/awx/blob/devel/docs/build\_awx\_image.md](https://github.com/ansible/awx/blob/devel/docs/build_awx_image.md) Tin ---
**Author:** ![Wei-Yen\_Tan](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/wei-yen_tan/32/4111_2.png) [@Wei-Yen\_Tan](https://forum.ansible.com/u/Wei-Yen_Tan)\ **Post date:** [September 17, 2021, 4:13am UTC](https://forum.ansible.com/t/running-two-versions-of-awx-in-the-same-k8s-cluster/35566/13 "2021-09-17T04:13:50Z")
What tools do you need to add? Awx19 uses execution environments now which runs in a container. ---
**Author:** ![tin](https://avatars.discourse-cdn.com/v4/letter/t/5f9b8f/32.png) [@tin](https://forum.ansible.com/u/tin)\ **Post date:** [September 17, 2021, 4:21am UTC](https://forum.ansible.com/t/running-two-versions-of-awx-in-the-same-k8s-cluster/35566/14 "2021-09-17T04:21:17Z")
I can’t speak for Cnu, but I built my own AWX images for security purposes. I also build custom EE images for security, and to include additional + custom roles. Tin ---
**Author:** ![Wei-Yen\_Tan](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/wei-yen_tan/32/4111_2.png) [@Wei-Yen\_Tan](https://forum.ansible.com/u/Wei-Yen_Tan)\ **Post date:** [September 17, 2021, 4:23am UTC](https://forum.ansible.com/t/running-two-versions-of-awx-in-the-same-k8s-cluster/35566/15 "2021-09-17T04:23:08Z")
What security things did you put in it? Curious to know ---
**Author:** ![tin](https://avatars.discourse-cdn.com/v4/letter/t/5f9b8f/32.png) [@tin](https://forum.ansible.com/u/tin)\ **Post date:** [September 17, 2021, 5:23am UTC](https://forum.ansible.com/t/running-two-versions-of-awx-in-the-same-k8s-cluster/35566/16 "2021-09-17T05:23:43Z")
Have you heard of SolarWinds supply chain attack? The reason I built my own images is so all the components are pulled from our internal repo. They have already been scanned by our tools for security vulnerabilities. By building with our tool chain, using libraries and components from our repo, we can be more confident that they are safe to use. Tin ---
**Author:** ![Wei-Yen\_Tan](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/wei-yen_tan/32/4111_2.png) [@Wei-Yen\_Tan](https://forum.ansible.com/u/Wei-Yen_Tan)\ **Post date:** [September 17, 2021, 5:28am UTC](https://forum.ansible.com/t/running-two-versions-of-awx-in-the-same-k8s-cluster/35566/17 "2021-09-17T05:28:46Z")
Good call. Something to consider ---
**Author:** ![Cnu\_k](https://avatars.discourse-cdn.com/v4/letter/c/a88e4f/32.png) [@Cnu\_k](https://forum.ansible.com/u/Cnu_k)\ **Post date:** [September 17, 2021, 1:08pm UTC](https://forum.ansible.com/t/running-two-versions-of-awx-in-the-same-k8s-cluster/35566/18 "2021-09-17T13:08:51Z")
Thank you Tin for the repo information and your approach to build custom images to reduce what’s getting into the network. We’re using Artifactory as a container registry and for storing all other binary artifacts as well. We’re trying to restrict external images but not there yet. I will try to find custom docker build image for the awx\_ee containers as well and give that a try. Thanks ---
**Author:** ![Wei-Yen\_Tan](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/wei-yen_tan/32/4111_2.png) [@Wei-Yen\_Tan](https://forum.ansible.com/u/Wei-Yen_Tan)\ **Post date:** [September 17, 2021, 1:10pm UTC](https://forum.ansible.com/t/running-two-versions-of-awx-in-the-same-k8s-cluster/35566/19 "2021-09-17T13:10:02Z")
You can write your own