# AWX, ansbile-callback not working **URL:** \ **Category:** AWX Project\ **Tags:** kubernetes, awx\ **Created:** [August 24, 2023, 8:53am UTC](https://forum.ansible.com/t/awx-ansbile-callback-not-working/38366 "2023-08-24T08:53:27Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![Jurgen\_Ponds](https://avatars.discourse-cdn.com/v4/letter/j/7ab992/32.png) [@Jurgen\_Ponds](https://forum.ansible.com/u/Jurgen_Ponds)\ **Post date:** [August 24, 2023, 8:53am UTC](https://forum.ansible.com/t/awx-ansbile-callback-not-working/38366/1 "2023-08-24T08:53:27Z")
Hello, I’am using AWX 22.7.0 on a kubernetes environment. Kubernetes is using MetalLB and Ingress. AWX is configured to use ingress in with type loadbalancer. Everything is working fine, expect the ansible-callback. As seen many time: /usr/bin/curl -k -s -H ‘Content-Type: application/json’ --data ‘{“host\_config\_key”:“#######”}’ [https://awx.example.com/api/v2/job\_templates/11/callback/](https://awx.example.com/api/v2/job_templates/11/callback/) {“msg”:“No matching host could be found!”} Solutions provided are the setting of: HTTP\_X\_FORWARDED\_FOR, but this does not work for me. Can anybody help me out? Kind regards Jurgen Ponds ---
**Author:** ![AWX\_Project](https://avatars.discourse-cdn.com/v4/letter/a/eada6e/32.png) [@AWX\_Project](https://forum.ansible.com/u/AWX_Project)\ **Post date:** [August 25, 2023, 5:58pm UTC](https://forum.ansible.com/t/awx-ansbile-callback-not-working/38366/2 "2023-08-25T17:58:14Z")
can you confirm that you added “HTTP\_X\_FORWARDED\_FOR” in settings \> Miscellaneous System Settings \> Remote Host Headers? Can you also confirm that the value you sent for HTTP\_X\_FORWARDED\_FOR matches a host in your inventory? you can read more here [https://docs.ansible.com/ansible-tower/latest/html/installandreference/proxy-support.html](https://docs.ansible.com/ansible-tower/latest/html/installandreference/proxy-support.html) AWX Team ---
**Author:** ![Jurgen\_Ponds](https://avatars.discourse-cdn.com/v4/letter/j/7ab992/32.png) [@Jurgen\_Ponds](https://forum.ansible.com/u/Jurgen_Ponds)\ **Post date:** [August 29, 2023, 9:09am UTC](https://forum.ansible.com/t/awx-ansbile-callback-not-working/38366/3 "2023-08-29T09:09:46Z")
Jurgen Ponds ongelezen, 11:07 (nu)    aan [awx-p...@googlegroups.com](mailto:awx-p...@googlegroups.com) Hello Yes I can confirm the setting is done. ![image.png](https://groups.google.com/group/awx-project/attach/15486fd320d8fa/image.png?part=0.1&view=1) I can not find any relation between the hostname/ipaddress of the server i the logging of the awx-web container [pid: 39|app: 0|req: 3/34] 10.244.0.1 () {54 vars in 783 bytes} [Tue Aug 29 09:00:23 2023] POST /api/v2/job\_templates/11/callback/ =\> generated 42 bytes in 38915 msecs (HTTP/1.1 400) 11 headers in 394 bytes (1 switches on core 0) 2023-08-29 09:03:19,442 INFO [78848aea4228483189dcd833c35bb3fe] awx.analytics.job\_lifecycle inventoryupdate-439 created 2023-08-29 09:03:43,660 WARNING [78848aea4228483189dcd833c35bb3fe] awx.api.generics status 400 received by user AnonymousUser attempting to access /api/v2/job\_templates/11/callback/ from 10.244.0.1 2023-08-29 09:03:43,662 WARNING [78848aea4228483189dcd833c35bb3fe] django.request Bad Request: /api/v2/job\_templates/11/callback/ 2023-08-29 09:03:43,662 WARNING [78848aea4228483189dcd833c35bb3fe] django.request Bad Request: /api/v2/job\_templates/11/callback/ [pid: 35|app: 0|req: 15/35] 10.244.0.1 () {54 vars in 783 bytes} [Tue Aug 29 09:03:18 2023] POST /api/v2/job\_templates/11/callback/ =\> generated 42 bytes in 24683 msecs (HTTP/1.1 400) 11 headers in 394 bytes (1 switches on core 0) 10.244.0.1 - - [29/Aug/2023:09:03:43 +0000] “POST /api/v2/job\_templates/11/callback/ HTTP/1.1” 400 42 “-” “curl/7.76.1” “10.244.0.1” 10.244.0.1 - - [29/Aug/2023:09:04:12 +0000] “GET /websocket/ HTTP/1.1” 101 1153 “-” “Mozilla/5.0 (X11; Linux x86\_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/[116.0.0.0](http://116.0.0.0/) Safari/537.36 OPR/[102.0.0.0](http://102.0.0.0/)” “10.244.0.1” [pid: 38|app: 0|req: 8/36] 10.244.0.1 () {74 vars in 1337 bytes} [Tue Aug 29 09:04:12 2023] OPTIONS /api/v2/settings/all/ =\> generated 330064 bytes in 773 msecs (HTTP/1.1 200) 14 headers in 601 bytes (1 switches on core 0) 10.244.0.1 - - [29/Aug/2023:09:04:13 +0000] “OPTIONS /api/v2/settings/all/ HTTP/1.1” 200 330064 “[https://awx.jpp.nl/](https://awx.jpp.nl/)” “Mozilla/5.0 (X11; Linux x86\_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/[116.0.0.0](http://116.0.0.0/) Safari/537.36 OPR/[102.0.0.0](http://102.0.0.0/)” “10.244.0.1” 10.244.0.1 - - [29/Aug/2023:09:04:13 +0000] “GET /static/media/default.strings.json HTTP/1.1” 200 53 “[https://awx.jpp.nl/](https://awx.jpp.nl/)” “Mozilla/5.0 (X11; Linux x86\_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/[116.0.0.0](http://116.0.0.0/) Safari/537.36 OPR/[102.0.0.0](http://102.0.0.0/)” “10.244.0.1” 10.244.0.1 - - [29/Aug/2023:09:04:13 +0000] “GET /static/media/default.strings.json HTTP/1.1” 200 53 “[https://awx.jpp.nl/](https://awx.jpp.nl/)” “Mozilla/5.0 (X11; Linux x86\_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/[116.0.0.0](http://116.0.0.0/) Safari/537.36 OPR/[102.0.0.0](http://102.0.0.0/)” “10.244.0.1” 10.244.0.1 - - [29/Aug/2023:09:04:14 +0000] “GET /api/v2/settings/system/ HTTP/1.1” 200 1022 “[https://awx.jpp.nl/](https://awx.jpp.nl/)” “Mozilla/5.0 (X11; Linux x86\_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/[116.0.0.0](http://116.0.0.0/) Safari/537.36 OPR/[102.0.0.0](http://102.0.0.0/)” “10.244.0.1” [pid: 35|app: 0|req: 16/37] 10.244.0.1 () {72 vars in 1306 bytes} [Tue Aug 29 09:04:14 2023] GET /api/v2/settings/system/ =\> generated 1022 bytes in 121 msecs (HTTP/1.1 200) 14 headers in 599 bytes (1 switches on core 0) The IP Address should be 10.0.0.11, hostname [lightning.jpp.nl](http://lightning.jpp.nl/) The callback command is:/usr/bin/curl -k -s --data “host\_config\_key=14061972” [https://awx.jpp.nl/api/v2/job\_templates/11/callback/](https://awx.jpp.nl/api/v2/job_templates/11/callback/) Kind regards Jurgen ---
**Author:** ![Jurgen\_Ponds](https://avatars.discourse-cdn.com/v4/letter/j/7ab992/32.png) [@Jurgen\_Ponds](https://forum.ansible.com/u/Jurgen_Ponds)\ **Post date:** [August 29, 2023, 9:11am UTC](https://forum.ansible.com/t/awx-ansbile-callback-not-working/38366/4 "2023-08-29T09:11:18Z")
![image.png](https://us1.discourse-cdn.com/ansible/original/2X/f/ffc5bc6ba574e4c1c63240987027f9363a360a94.png) ---
**Author:** ![AWX\_Project](https://avatars.discourse-cdn.com/v4/letter/a/eada6e/32.png) [@AWX\_Project](https://forum.ansible.com/u/AWX_Project)\ **Post date:** [August 30, 2023, 7:00pm UTC](https://forum.ansible.com/t/awx-ansbile-callback-not-working/38366/5 "2023-08-30T19:00:31Z")
POST /api/v2/job\_templates/11/callback/ =\> generated 42 bytes in 24683 msecs (HTTP/1.1 400) 11 headers in 394 bytes (1 switches on core 0) So what are the headers that are being sent to this endpoint? were you able to grab those? AWX Team ---
**Author:** ![Jurgen\_Ponds](https://avatars.discourse-cdn.com/v4/letter/j/7ab992/32.png) [@Jurgen\_Ponds](https://forum.ansible.com/u/Jurgen_Ponds)\ **Post date:** [August 30, 2023, 7:43pm UTC](https://forum.ansible.com/t/awx-ansbile-callback-not-working/38366/6 "2023-08-30T19:43:29Z")
What I sent is the curl command. /usr/bin/curl -k -s --data “host\_config\_key=14061972” [https://awx.jpp.nl/api/v2/job\_templates/11/callback/](https://awx.jpp.nl/api/v2/job_templates/11/callback/) This is what is listed the kickstart file generated by foreman/katello kind regards Jurgen ---
**Author:** ![AWX\_Project](https://avatars.discourse-cdn.com/v4/letter/a/eada6e/32.png) [@AWX\_Project](https://forum.ansible.com/u/AWX_Project)\ **Post date:** [September 6, 2023, 6:39pm UTC](https://forum.ansible.com/t/awx-ansbile-callback-not-working/38366/7 "2023-09-06T18:39:31Z")
so that curl command is not sending HTTP\_X\_FORWARDED\_FOR header at all? is there a way to customize what foreman/katello sends? ---
**Author:** ![Jurgen\_Ponds](https://avatars.discourse-cdn.com/v4/letter/j/7ab992/32.png) [@Jurgen\_Ponds](https://forum.ansible.com/u/Jurgen_Ponds)\ **Post date:** [September 7, 2023, 7:50am UTC](https://forum.ansible.com/t/awx-ansbile-callback-not-working/38366/8 "2023-09-07T07:50:33Z")
Yes, that is possible. What do I need to add? I’m not so familiar with this stuff kind regards Jurgen ---
**Author:** ![Jurgen\_Ponds](https://avatars.discourse-cdn.com/v4/letter/j/7ab992/32.png) [@Jurgen\_Ponds](https://forum.ansible.com/u/Jurgen_Ponds)\ **Post date:** [September 7, 2023, 12:17pm UTC](https://forum.ansible.com/t/awx-ansbile-callback-not-working/38366/9 "2023-09-07T12:17:35Z")
Ok, I already found out. And that works fine. /usr/bin/curl -k -s --header “X-Forwarded-For: ” --data “host\_config\_key=” [https://foreman/api/v2/job\_templates/11/callback/](https://foreman/api/v2/job_templates/11/callback/) ---
**Author:** ![Jurgen\_Ponds](https://avatars.discourse-cdn.com/v4/letter/j/7ab992/32.png) [@Jurgen\_Ponds](https://forum.ansible.com/u/Jurgen_Ponds)\ **Post date:** [September 8, 2023, 9:39am UTC](https://forum.ansible.com/t/awx-ansbile-callback-not-working/38366/10 "2023-09-08T09:39:55Z")
mmm, I’m using the same configuration somewhere else in a kubernetes cluster and there the x-forwarded-for header does not work. ☹ On a single node kubernetes it does work. In both cases I’m using ingress, but in the awx-web container I don’t see any external ip addresses, only internal kubernetes IP addresses ---
**Author:** ![Jurgen\_Ponds](https://avatars.discourse-cdn.com/v4/letter/j/7ab992/32.png) [@Jurgen\_Ponds](https://forum.ansible.com/u/Jurgen_Ponds)\ **Post date:** [September 15, 2023, 8:54am UTC](https://forum.ansible.com/t/awx-ansbile-callback-not-working/38366/11 "2023-09-15T08:54:39Z")
Hello I was a bit mislead, because it looked like the callback was working, but it turned out it was starting the jobtemplate for the kubernetes node in stead of the node which did the callback action. So it still does not work with the X-FORWARDED\_FOR in the curl header.