Custom DNS server in resolv.conf for the AWX job container

I am using AWX installation in K8s using AWX operator. As part of of our job template, we need to pull some files from an external private git repo. Git repo is a private URL resolvable by a private custom DNS server. For web and task pod we have added the DNS server IP to our dnsPolicy attribute of the deployment. The git repo is reachable when we exec into the task pod, awx-tower-ee container and do the pull. However when the job template is launched, it failed with “not able to resolve” the host name. From the documents and google search i found that the job ran with an ephemeral container similar to awx-tower-ee container however it does not inherit the network policies including DNS server. I have tested it by adding a task to print the resolv.conf from the job container and found that the nameserver is missing.


- name: Show /etc/resolv.conf
      command: cat /etc/resolv.conf
      register: resolv_conf
    - debug:
        var: resolv_conf.stdout_lines

TASK [debug] *******************************************************************
ok: [localhost] => {
    "resolv_conf.stdout_lines": [
        "search awx.svc.cluster.local svc.cluster.local cluster.local us-central1-c.c.hclsw-gcp-o11y.internal c.hclsw-gcp-o11y.internal google.internal",
        "nameserver 34.118.224.10",
        "options ndots:5"
    ]
}

resolv.conf entry in the task pod (awx-tower-ee container)

search awx.svc.cluster.local svc.cluster.local cluster.local us-central1-b.c.hclsw-gcp-o11y.internal c.hclsw-gcp-o11y.internal google.internal
nameserver 34.118.224.10
nameserver 10.**.**.**
options ndots:5

I tried changing the resolv.conf file by copy task, however it seems like file is not writable.

    - name: Set custom DNS server
      ansible.builtin.copy:
        dest: /etc/resolv.conf
        content: "nameserver {{ target_dns_server }}\nsearch your_domain.com\n"
      become: true
TASK [Set custom DNS server] ***************************************************
fatal: [localhost]: FAILED! => {"msg": "Failed to get information on remote file (/etc/resolv.conf): /bin/sh: line 1: sudo: command not found\n"}

We are tried creating custom EE image however even then the file seems to be not writable there. the docker build image fails when we try to edit the file to add the nameserver


10 |     # Append the additional DNS server
  11 | >>> RUN echo "nameserver 10.**.**.**" >> /etc/resolv.conf
  12 |
  13 |     # Switch back to default AWX EE user
--------------------
ERROR: failed to solve: process "/bin/sh -c echo \"nameserver 10.134.49.13\" >> /etc/resolv.conf" did not complete successfully: exit code: 1

Is there anyway now I can use a custom DNS nameserver as part of the job container so that i am able to reach the private endpoints resolvable through custom DNS ?