AWX Job scheduling issue

hello !

would anyone have an idea what could cause the awx task pods to take upwards of ~10 minutes to launch a job in kubernetes (container groups)?

in my setup you can assume there’s:

  • 4 web pods
  • 6 task pods (limits = 1CPU, 8Gi )

e.g logs from the task pod (task container)

│ 2026-01-29 02:22:40,607 INFO     [-] awx.main.wsrelay Starting producer for 172.22.14.47-job_events-408215                                                                                                    │
│ 2026-01-29 02:22:40,607 INFO     [-] awx.main.wsrelay Starting producer for 172.22.14.47-jobs-summary                                                                                                         │
│ 2026-01-29 02:23:16,180 INFO     [-] awx.main.wsrelay Producer 172.22.14.47-control-limit_reached_1 has no subscribers, shutting down.                                                                        │
│ 2026-01-29 02:23:16,181 INFO     [-] awx.main.wsrelay Producer 172.22.14.47-jobs-status_changed has no subscribers, shutting down.                                                                            │
│ 2026-01-29 02:23:20,612 INFO     [-] awx.main.wsrelay Producer 172.22.14.47-jobs-summary has no subscribers, shutting down.                                                                                   │
│ 2026-01-29 02:23:20,612 INFO     [-] awx.main.wsrelay Producer 172.22.14.47-job_events-408215 has no subscribers, shutting down.                                                                              │
│ 2026-01-29 02:23:20,779 INFO     [-] awx.main.wsrelay Starting producer for 172.22.14.47-jobs-status_changed                                                                                                  │
│ 2026-01-29 02:23:20,779 INFO     [-] awx.main.wsrelay Starting producer for 172.22.14.47-project_update_events-408217                                                                                         │
│ 2026-01-29 02:23:20,779 INFO     [-] awx.main.wsrelay Starting producer for 172.22.14.47-jobs-summary                                                                                                         │
│ 2026-01-29 02:23:56,813 INFO     [-] awx.main.wsrelay Starting producer for 172.22.12.133-project_update_events-408217                                                                                        │
│ 2026-01-29 02:23:56,813 INFO     [-] awx.main.wsrelay Starting producer for 172.22.12.133-jobs-summary                                                                                                        │
│ 2026-01-29 02:24:00,783 INFO     [-] awx.main.wsrelay Producer 172.22.14.47-jobs-summary has no subscribers, shutting down.                                                                                   │
│ 2026-01-29 02:24:00,783 INFO     [-] awx.main.wsrelay Producer 172.22.14.47-project_update_events-408217 has no subscribers, shutting down.                                                                   │
│ 2026-01-29 02:24:00,783 INFO     [-] awx.main.wsrelay Producer 172.22.14.47-jobs-status_changed has no subscribers, shutting down.                                                                            │
│ 2026-01-29 02:24:23,277 INFO     [-] awx.main.wsrelay Starting producer for 172.22.14.47-jobs-status_changed                                                                                                  │
│ 2026-01-29 02:24:23,277 INFO     [-] awx.main.wsrelay Starting producer for 172.22.14.47-control-limit_reached_1                                                                                              │
│ 2026-01-29 02:24:24,603 INFO     [-] awx.main.wsrelay Starting producer for 172.22.12.133-job_events-408215                                                                                                   │
│ 2026-01-29 02:24:31,537 INFO     [-] awx.main.wsrelay Producer 172.22.10.1-jobs-summary has no subscribers, shutting down.                                                                                    │
│ 2026-01-29 02:24:31,537 INFO     [-] awx.main.wsrelay Producer 172.22.10.1-job_events-408215 has no subscribers, shutting down.                                                                               │
│ 2026-01-29 02:24:54,607 INFO     [-] awx.main.wsrelay Producer 172.22.12.133-job_events-408215 has no subscribers, shutting down.

UI view