Could not resolve host: mirrors.centos.org during the build of the docker container image

Archives AWX Project
1

I’m trying to build our own docker container image with:

su - awx
git clone -b 22.3.0 https://github.com/ansible/awx.git
cd awx
git switch -c release_4.4
. /opt/awx/venv/bin/activate
make docker-compose-build

which leads to:

ansible-playbook tools/ansible/dockerfile.yml
-e dockerfile_name=Dockerfile.dev
-e build_dev=True
-e receptor_image=quay.io/ansible/receptor:devel
[WARNING]: No inventory was parsed, only implicit localhost is available
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match ‘all’

PLAY [Render AWX Dockerfile and sources] ************************************************************************************************************************************

TASK [Gathering Facts] ******************************************************************************************************************************************************
ok: [localhost]

TASK [dockerfile : Create _build directory] *********************************************************************************************************************************
ok: [localhost]

TASK [dockerfile : Render supervisor configs] *******************************************************************************************************************************
ok: [localhost] => (item=supervisor_web.conf)
ok: [localhost] => (item=supervisor_task.conf)
ok: [localhost] => (item=supervisor_rsyslog.conf)

TASK [dockerfile : Render Dockerfile] ***************************************************************************************************************************************
changed: [localhost]

PLAY RECAP ******************************************************************************************************************************************************************
localhost : ok=4 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0

DOCKER_BUILDKIT=1 docker build
-f Dockerfile.dev
-t ghcr.io/ansible/awx_devel:release_4.4
–build-arg BUILDKIT_INLINE_CACHE=1
–cache-from=ghcr.io/ansible/awx_devel:release_4.4 .
[+] Building 75.7s (11/47)
=> [internal] load build definition from Dockerfile.dev 0.0s
=> => transferring dockerfile: 7.57kB 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 56B 0.0s
=> [internal] load metadata for quay.io/ansible/receptor:devel 0.4s
=> [internal] load metadata for quay.io/centos/centos:stream9 0.4s
=> importing cache manifest from ghcr.io/ansible/awx_devel:release_4.4 0.8s
=> [builder 1/10] FROM quay.io/centos/centos:stream9@sha256:d075b8cb028de107de53d601512f6f2fc70c8d3c1313959d5925a5386dd1d665 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 47.03kB 0.0s
=> FROM quay.io/ansible/receptor:devel@sha256:b14b7bc6a77fa04e61fd550bdec0e4134aedba23a4144bcd13ce738523c6acf9 0.0s
=> CACHED [builder 2/10] RUN rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial 0.0s
=> ERROR [builder 3/10] RUN dnf -y update && dnf install -y ‘dnf-command(config-manager)’ && dnf config-manager --set-enabled crb && dnf -y install gcc 74.4s
=> ERROR [stage-1 3/32] RUN dnf -y update && dnf install -y ‘dnf-command(config-manager)’ && dnf config-manager --set-enabled crb && dnf -y install acl 74.3s

2

Same issue with:

  • awx_devel:release_4.2
  • awx_devel:release_4.3
3

I have no such issue using a CentOS stream 9 host instead of Ubuntu.

Any idea why?

4

We see this error and seems relevant to your issue
#0 74.30 - Curl error (6): Couldn’t resolve host name for https://mirrors.centos.org/metalink?repo=centos-baseos-9-stream&arch=x86_64&protocol=https,http [Could not resolve host: mirrors.centos.org]

are you able to successfully curl this url from the terminal that you are running the make docker-compose-build https://mirrors.centos.org/metalink?repo=centos-baseos-9-stream&arch=x86_64&protocol=https,http

AWX Team

5

Yes:

(venv) awx@sdx1-kvm:~/git-awx$ curl https://mirrors.centos.org/metalink?repo=centos-baseos-9-stream&arch=x86_64&protocol=https,http
[1] 137878
[2] 137879
(venv) awx@sdx1-kvm:~/git-awx$ <?xml version="1.0" encoding="utf-8"?>

[1]- Done curl https://mirrors.centos.org/metalink?repo=centos-baseos-9-stream
[2]+ Done arch=x86_64
(venv) awx@sdx1-kvm:~/git-awx$

The issue is still there when ‘make docker-compose-build’ is run as awx (non-root) user.
Here is some issue(s) which appear in the docker log:

echo ‘q’ | sudo systemctl --no-pager --full status docker

● docker.service - Docker Application Container Engine
Loaded: loaded (/lib/systemd/system/docker.service; enabled; preset: enabled)
Active: active (running) since Mon 2023-06-05 11:42:24 UTC; 14min ago
TriggeredBy: ● docker.socket
Docs: https://docs.docker.com
Main PID: 59654 (dockerd)
Tasks: 12
Memory: 216.0M
CPU: 3.006s
CGroup: /system.slice/docker.service
└─59654 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock

Jun 05 11:50:58 sdx1-kvm dockerd[59654]: time=“2023-06-05T11:50:58.721767833Z” level=debug msg=“2023/06/05 11:50:58 WARNING: [core] [Channel #11 SubChannel #12] grpc: addrConn.createTransport failed to connect to {” library=grpc
Jun 05 11:50:58 sdx1-kvm dockerd[59654]: time=“2023-06-05T11:50:58.721808791Z” level=debug msg=" "Addr": "localhost"," library=grpc
Jun 05 11:50:58 sdx1-kvm dockerd[59654]: time=“2023-06-05T11:50:58.721813951Z” level=debug msg=" "ServerName": "localhost"," library=grpc
Jun 05 11:50:58 sdx1-kvm dockerd[59654]: time=“2023-06-05T11:50:58.721817898Z” level=debug msg=" "Attributes": null," library=grpc
Jun 05 11:50:58 sdx1-kvm dockerd[59654]: time=“2023-06-05T11:50:58.721821385Z” level=debug msg=" "BalancerAttributes": null," library=grpc
Jun 05 11:50:58 sdx1-kvm dockerd[59654]: time=“2023-06-05T11:50:58.721824711Z” level=debug msg=" "Type": 0," library=grpc
Jun 05 11:50:58 sdx1-kvm dockerd[59654]: time=“2023-06-05T11:50:58.721828698Z” level=debug msg=" "Metadata": null" library=grpc
Jun 05 11:50:58 sdx1-kvm dockerd[59654]: time=“2023-06-05T11:50:58.721831995Z” level=debug msg=“}. Err: connection error: desc = "transport: Error while dialing only one connection allowed"” library=grpc
Jun 05 11:52:33 sdx1-kvm dockerd[59654]: time=“2023-06-05T11:52:33.724284994Z” level=warning msg=“leaked execError detected and released”

6

Any suggestion regarding this “Error while dialing only one connection allowed” with grpc?

7

can you provide information about where you are running the make command? is this on a linux VM? if so which OS and version, and which docker version are you using?

AWX Team

8
  • Ubuntu 23.04 lunar on a KVM VM running on a similar host
  • docker-ce 5:24.0.2-1~ubuntu.23.04~lunar from the official docker PPA (deb [arch=amd64] https://download.docker.com/linux/ubuntu lunar stable edge)
  • python3.9 3.9.5-3ubuntu0~21.04.1 (backported from Ubuntu hirsute 21.04)
  • pip/wheel/ansible/docker/docker-compose/redis/setuptools-scm installed from PyPi using pip3 inside a virtual env
9

Exact same issue with:

Same strange grpc error:

echo q | systemctl --no-pager --full status docker

● docker.service - Docker Application Container Engine
Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
Active: active (running) since Wed 2023-06-14 15:43:04 UTC; 11min ago
TriggeredBy: ● docker.socket
Docs: https://docs.docker.com
Main PID: 211829 (dockerd)
Tasks: 12
Memory: 214.3M
CPU: 4.695s
CGroup: /system.slice/docker.service
└─211829 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock

Jun 14 15:45:59 sdx1-kvm dockerd[211829]: time=“2023-06-14T15:45:59.935797363Z” level=debug msg=“ReleaseAddress(LocalDefault/172.17.0.0/16, 172.17.0.3)”
Jun 14 15:45:59 sdx1-kvm dockerd[211829]: time=“2023-06-14T15:45:59.935818162Z” level=debug msg=“Released address Address:172.17.0.3 Sequence:Bits: 65536, Unselected: 65533, Sequence: (0xc0000000, 1)->(0x0, 2046)->(0x1, 1)->end Curr:5”
Jun 14 15:46:03 sdx1-kvm dockerd[211829]: time=“2023-06-14T15:46:03.897765133Z” level=debug msg=“2023/06/14 15:46:03 WARNING: [core] [Channel #11 SubChannel #12] grpc: addrConn.createTransport failed to connect to {” library=grpc
Jun 14 15:46:03 sdx1-kvm dockerd[211829]: time=“2023-06-14T15:46:03.903195475Z” level=debug msg=" "Addr": "localhost"," library=grpc
Jun 14 15:46:03 sdx1-kvm dockerd[211829]: time=“2023-06-14T15:46:03.903237455Z” level=debug msg=" "ServerName": "localhost"," library=grpc
Jun 14 15:46:03 sdx1-kvm dockerd[211829]: time=“2023-06-14T15:46:03.903246712Z” level=debug msg=" "Attributes": null," library=grpc
Jun 14 15:46:03 sdx1-kvm dockerd[211829]: time=“2023-06-14T15:46:03.903254036Z” level=debug msg=" "BalancerAttributes": null," library=grpc
Jun 14 15:46:03 sdx1-kvm dockerd[211829]: time=“2023-06-14T15:46:03.903261229Z” level=debug msg=" "Type": 0," library=grpc
Jun 14 15:46:03 sdx1-kvm dockerd[211829]: time=“2023-06-14T15:46:03.903273694Z” level=debug msg=" "Metadata": null" library=grpc
Jun 14 15:46:03 sdx1-kvm dockerd[211829]: time=“2023-06-14T15:46:03.903280757Z” level=debug msg=“}. Err: connection error: desc = "transport: Error while dialing only one connection allowed"” library=grpc