# Awx-operator and Karpenter in EKS **URL:** \ **Category:** Get Help\ **Tags:** kubernetes, awx-operator, awx, collections\ **Created:** [November 21, 2023, 9:09pm UTC](https://forum.ansible.com/t/awx-operator-and-karpenter-in-eks/2498 "2023-11-21T21:09:25Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![trippinnik](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/trippinnik/32/4308_2.png) [@trippinnik](https://forum.ansible.com/u/trippinnik)\ **Post date:** [November 21, 2023, 9:09pm UTC](https://forum.ansible.com/t/awx-operator-and-karpenter-in-eks/2498/1 "2023-11-21T21:09:25Z")
```auto Failed to schedule pod, incompatible with provisioner "default", did not tolerate /karpenter=:NoSchedule; incompatible with provisioner "default-spot", did not tolerate /karpenter=:NoSchedule ``` Does awx-operator work with Karpenter or any plans to? ---
**Author:** ![jmack](https://avatars.discourse-cdn.com/v4/letter/j/57b2e6/32.png) [@jmack](https://forum.ansible.com/u/jmack)\ **Post date:** [November 22, 2023, 8:07pm UTC](https://forum.ansible.com/t/awx-operator-and-karpenter-in-eks/2498/2 "2023-11-22T20:07:14Z")
Hello @trippinnik, To my knowledge we haven’t tested awx-operator with Karpenter. If you try this out, please let us know how it goes and share back any pain points you run in to. You’ll probably want to be mindful of what you set node\_selector, task\_affinity, and web\_affinity to. The defaults will not discriminate which node the pods are scheduled on. ---
**Author:** ![trippinnik](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/trippinnik/32/4308_2.png) [@trippinnik](https://forum.ansible.com/u/trippinnik)\ **Post date:** [November 22, 2023, 9:36pm UTC](https://forum.ansible.com/t/awx-operator-and-karpenter-in-eks/2498/3 "2023-11-22T21:36:06Z")
The error is above when trying to schedule a pod, so I guess not. I don’t care what nodes things run on but it it would be ideal if Karpenter knew to start a node when the current nodes are full/used. The jobs are generally bursts of activity and then quiet ---
**Author:** ![alvaroc20](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/alvaroc20/32/3485_2.png) [@alvaroc20](https://forum.ansible.com/u/alvaroc20)\ **Post date:** [November 2, 2024, 11:26pm UTC](https://forum.ansible.com/t/awx-operator-and-karpenter-in-eks/2498/4 "2024-11-02T23:26:59Z")
Hi! I’m running AWX with Karpenter. Do you need any help about this or were you able to resolve it? I just saw your message now, but it’s been a while. ---
**Author:** ![deanso](https://avatars.discourse-cdn.com/v4/letter/d/94ad74/32.png) [@deanso](https://forum.ansible.com/u/deanso)\ **Post date:** [November 22, 2024, 10:11am UTC](https://forum.ansible.com/t/awx-operator-and-karpenter-in-eks/2498/5 "2024-11-22T10:11:56Z")
@alvaroc20 , How did you get this working? We use helm charts to set this up and want to run the awx-task pods on a seperate karpenter nodepool, but our pods don’t get scheduled on the karpenter nodepool. ---
**Author:** ![alvaroc20](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/alvaroc20/32/3485_2.png) [@alvaroc20](https://forum.ansible.com/u/alvaroc20)\ **Post date:** [November 22, 2024, 11:14am UTC](https://forum.ansible.com/t/awx-operator-and-karpenter-in-eks/2498/6 "2024-11-22T11:14:42Z")
Hi @deanso In my case, I use a single NodePool for both `task` and `web`. If you could share the specific error you’re encountering, I might be able to assist further. First and foremost, make sure that your Karpenter setup is fully operational. This includes **verifying that Karpenter is creating and scaling nodes** as expected. You can follow the official documentation to run some tests and confirm everything is working correctly. Once that’s in place, you’ll need to configure your `EC2NodeClass` and `NodePool` resources for Karpenter (these are typically managed as Kubernetes resources rather than directly through Helm). If you’ve got this ready and functional, you’re in a good position to proceed further! That said, based on what you mentioned, I believe you’d need two NodePools: one configured for `task` pods with a specific label referencing it, and another for `web` pods. In the AWX resource (CRD), you would need to make use of `task_node_selector` and `web_node_selector` to ensure the pods are scheduled where they need to be. Alternatively, you could consider using `nodeAffinity` or `taints/tolerations`, although the simplest approach might be `nodeSelector`. Be cautious about two key aspects: 1. **Architecture:** Ensure the image is for an `amd64` architecture. 2. **Resource Requests:** Pay close attention to the resource requests defined in your CRD for the pods, keeping in mind the instance types you’re using. I’ve noticed memory consumption is quite high, particularly for `web` pods. Here’s my configuration for reference: ```auto web_resource_requirements: limits: cpu: "2" memory: 2Gi requests: cpu: 100m memory: 1.5Gi task_resource_requirements: limits: cpu: "2" memory: 2Gi requests: cpu: 100m memory: 1Gi ``` I’m using **R-series** instances in AWS for this setup. If you can provide more information about the issue or where you’re currently stuck, I’d be happy to help further! ---
**Author:** ![deanso](https://avatars.discourse-cdn.com/v4/letter/d/94ad74/32.png) [@deanso](https://forum.ansible.com/u/deanso)\ **Post date:** [November 22, 2024, 5:50pm UTC](https://forum.ansible.com/t/awx-operator-and-karpenter-in-eks/2498/7 "2024-11-22T17:50:29Z")
> [@alvaroc20](#): > > First and foremost, make sure that your Karpenter setup is fully operational. This includes **verifying that Karpenter is creating and scaling nodes** as expected. You can follow the official documentation to run some tests and confirm everything is working correctly. @alvaroc20 , Thank you for your detailed response, really appreciated! I am currently not receiving any errors, but my AWX task and web pods are not being scheduled on the Karpenter node pool, but rather on the default Kubernetes nodes. So I think you are right, in that my Karpenter nodepool is unable to create nodes. I will look further into that, with the Karpenter docs. Thx! ---
**Author:** ![trippinnik](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/trippinnik/32/4308_2.png) [@trippinnik](https://forum.ansible.com/u/trippinnik)\ **Post date:** [November 26, 2024, 7:42pm UTC](https://forum.ansible.com/t/awx-operator-and-karpenter-in-eks/2498/9 "2024-11-26T19:42:22Z")
I have it working with below config: ```auto apiVersion: karpenter.sh/v1beta1 kind: NodePool metadata: name: awx-instances spec: disruption: consolidateAfter: 5m0s consolidationPolicy: WhenEmpty expireAfter: 168h0m0s limits: cpu: "16" memory: 2Ti template: metadata: labels: /karpenter-profile: awx spec: startupTaints: - key: ebs.csi.aws.com/agent-not-ready effect: NoExecute nodeClassRef: name: default requirements: - key: node.kubernetes.io/instance-type operator: In values: - r6a.xlarge - c6a.xlarge - c7a.xlarge - c6a.2xlarge - c5a.2xlarge - c6i.2xlarge - key: kubernetes.io/arch operator: In values: - amd64 - key: karpenter.sh/capacity-type operator: In values: - on-demand - key: kubernetes.io/os operator: In values: - linux taints: - effect: NoSchedule key: /karpenter ``` under spec: ```auto task_tolerations: | - effect: NoSchedule key: /karpenter task_node_selector: | /karpenter-profile: awx ``` your “unique\_name” between karpenter and the task\_tolerations and task\_node\_selector should match the karpenter profile. ---
**Author:** ![alvaroc20](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/alvaroc20/32/3485_2.png) [@alvaroc20](https://forum.ansible.com/u/alvaroc20)\ **Post date:** [November 28, 2024, 1:16pm UTC](https://forum.ansible.com/t/awx-operator-and-karpenter-in-eks/2498/10 "2024-11-28T13:16:25Z")
Hi @deanso Did you manage to get it working? Do you need help with anything? If you need help with Karpenter or AWX feel free to let me know.