Creating a docker image for execution environment AWX fails at Ansible-galaxy step

SaravanaSelvaraj · April 8, 2024, 3:12pm

docker build fails while running ansible-galaxy in the dockerfile.

Hello community please help me out here, I am trying to build a custom image from AWX-EE image using my dockerfile and it throws error the below error

Traceback (most recent call last):```
  File "/usr/local/bin/ansible-galaxy", line 5, in <module>
    from ansible.cli.galaxy import main
  File "/usr/local/lib/python3.9/site-packages/ansible/cli/__init__.py", line 103, in <module>
    from ansible.parsing.dataloader import DataLoader
  File "/usr/local/lib/python3.9/site-packages/ansible/parsing/dataloader.py", line 17, in <module>
    from ansible.module_utils.basic import is_executable
  File "/usr/local/lib/python3.9/site-packages/ansible/module_utils/basic.py", line 85, in <module>
    has_journal = hasattr(journal, 'sendv') and systemd_daemon.booted()
PermissionError: [Errno 1] Operation not permitted

My dockerfile is :

FROM quay.io/ansible/awx-ee:23.2.0
USER root
COPY requirements.yml requirements.yml
COPY requirements.txt requirements.txt
RUN python3 -m pip install --progress-bar off --upgrade pip && pip install --progress-bar off -r requirements.txt && ansible-galaxy collection install -r requirements.yml && rm -rf /root/.ansible /root/.cache /root/.pip
USER 1000

My requirement.txt is

pandas
requests

and the requirement.yml is

collections:

name: awx.awx

name: community.zabbix

name: google.cloud

name: community.general

name: ansible.posix

name: ansible.window

Please let me know what I am missing here ?

Thanks and Regards
Saravana Selvaraj

Denney-tech · April 8, 2024, 3:18pm

Are you running on an SELinux enabled machine? If so, I would try setting it to permissive mode and trying again.

SaravanaSelvaraj · April 8, 2024, 3:43pm

Thanks @Denney-tech for the response. I just figured it works with in another server with docker version 26.0.0.

BTW in my machine SELinux is not enabled from the machine where build fails :
[root@XXXXi]# sudo sestatus
SELinux status: disabled

And it is enabled from the machine where build is successful.

So Now I am wondering what is causing the issue , Docker version or SELinux ?

To figure it out I am installing lower version of docker in my success machine because I can do R&D only on that server.

Let me know your thoughts please ?

Denney-tech · April 8, 2024, 3:55pm

You’re getting a permissions error, so my first thought was SELinux (and is also a top google result), but if SELinux is disabled, that isn’t it.

My next thought is maybe user permissions, race conditions or installation path.

My AWX-EE build started with the one generated by ansible-builder, and the ansible-galaxy commands are their own RUN lines and look like this:

RUN ansible-galaxy role install $ANSIBLE_GALAXY_CLI_ROLE_OPTS -r requirements.yml --roles-path "/usr/share/ansible/roles"
RUN ANSIBLE_GALAXY_DISABLE_GPG_VERIFY=1 ansible-galaxy collection install $ANSIBLE_GALAXY_CLI_COLLECTION_OPTS -r requirements.yml --collections-path "/usr/share/ansible/collections"

Maybe the install path is important?

Edit: Just an observation of the error message, I’m curious as to why ansible-galaxy cares about the system journal or systemd_daemon.booted()…

SaravanaSelvaraj · April 8, 2024, 4:40pm

I ran again with below command

RUN python3 -m pip install --progress-bar off --upgrade pip && pip install --progress-bar off -r requirements.txt && ansible-galaxy collection install -r requirements.yml --collections-path “/usr/share/ansible/collections” && rm -rf /root/.ansible /root/.cache /root/.pip

Still same error !

kurokobo · April 9, 2024, 4:30am

Hi, I can’t reproduce your issue on my side with your files.
Does this error occur during a build? If so, please provide logs before and after errors as well.

SaravanaSelvaraj · April 9, 2024, 5:04am

Sure I can provide you the same. Yes it happens while building the image. It is not happening with me GCE server (docker version 24) but it happens with our on premise server (docker version 19)

I am running docker build . to build the image. Please assist me how can I generate logs before and after ?

kurokobo · April 9, 2024, 6:46am

Please assist me how can I generate logs before and after ?

Simply I want to see the full logs from docker build . command, not just the lines of error cut out of the logs.

However in the first place, Docker 19 is quite old and already reached EOL. In my opinion, considering upgrading Docker to newer version is more meaningful than trying to make docker build to work by keeping troubleshooting.

SaravanaSelvaraj · April 9, 2024, 7:45am

Hello, PFB for the detailed log

 docker build .
Sending build context to Docker daemon  4.608kB
Step 1/6 : FROM quay.io/ansible/awx-ee:23.2.0
 ---> 277e9aa01faf
Step 2/6 : USER root
 ---> Using cache
 ---> a121910fe93b
Step 3/6 : COPY requirements.yml requirements.yml
 ---> Using cache
 ---> 5f77a03a9858
Step 4/6 : COPY requirements.txt requirements.txt
 ---> Using cache
 ---> 17a4917e5e4a
Step 5/6 : RUN  python3 -m pip install --progress-bar off  --upgrade pip && pip install --progress-bar off  -r requirements.txt && ansible-galaxy collection install -r requirements.yml --collections-path "/usr/share/ansible/collections" && rm -rf /root/.ansible /root/.cache /root/.pip
 ---> Running in 30338431d242
Requirement already satisfied: pip in /usr/local/lib/python3.9/site-packages (23.2.1)
Collecting pip
  Obtaining dependency information for pip from https://files.pythonhosted.org/packages/8a/6a/19e9fe04fca059ccf770861c7d5721ab4c2aebc539889e97c7977528a53b/pip-24.0-py3-none-any.whl.metadata
  Downloading pip-24.0-py3-none-any.whl.metadata (3.6 kB)
Downloading pip-24.0-py3-none-any.whl (2.1 MB)
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 23.2.1
    Uninstalling pip-23.2.1:
      Successfully uninstalled pip-23.2.1
Successfully installed pip-24.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
Collecting pandas (from -r requirements.txt (line 1))
  Downloading pandas-2.2.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (19 kB)
Requirement already satisfied: requests in /usr/local/lib/python3.9/site-packages (from -r requirements.txt (line 2)) (2.31.0)
Collecting numpy<2,>=1.22.4 (from pandas->-r requirements.txt (line 1))
  Downloading numpy-1.26.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (61 kB)
Collecting python-dateutil>=2.8.2 (from pandas->-r requirements.txt (line 1))
  Downloading python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB)
Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.9/site-packages (from pandas->-r requirements.txt (line 1)) (2023.3.post1)
Collecting tzdata>=2022.7 (from pandas->-r requirements.txt (line 1))
  Downloading tzdata-2024.1-py2.py3-none-any.whl.metadata (1.4 kB)
Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib64/python3.9/site-packages (from requests->-r requirements.txt (line 2)) (3.2.0)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.9/site-packages (from requests->-r requirements.txt (line 2)) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.9/site-packages (from requests->-r requirements.txt (line 2)) (1.26.16)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.9/site-packages (from requests->-r requirements.txt (line 2)) (2023.7.22)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.9/site-packages (from python-dateutil>=2.8.2->pandas->-r requirements.txt (line 1)) (1.16.0)
Downloading pandas-2.2.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (13.0 MB)
Downloading numpy-1.26.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (18.2 MB)
Downloading python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB)
Downloading tzdata-2024.1-py2.py3-none-any.whl (345 kB)
Installing collected packages: tzdata, python-dateutil, numpy, pandas
  Attempting uninstall: python-dateutil
    Found existing installation: python-dateutil 2.8.1
    Uninstalling python-dateutil-2.8.1:
      Successfully uninstalled python-dateutil-2.8.1
Successfully installed numpy-1.26.4 pandas-2.2.1 python-dateutil-2.9.0.post0 tzdata-2024.1
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
Traceback (most recent call last):
  File "/usr/local/bin/ansible-galaxy", line 5, in <module>
    from ansible.cli.galaxy import main
  File "/usr/local/lib/python3.9/site-packages/ansible/cli/__init__.py", line 103, in <module>
    from ansible.parsing.dataloader import DataLoader
  File "/usr/local/lib/python3.9/site-packages/ansible/parsing/dataloader.py", line 17, in <module>
    from ansible.module_utils.basic import is_executable
  File "/usr/local/lib/python3.9/site-packages/ansible/module_utils/basic.py", line 85, in <module>
    has_journal = hasattr(journal, 'sendv') and systemd_daemon.booted()
PermissionError: [Errno 1] Operation not permitted
The command '/bin/sh -c python3 -m pip install --progress-bar off  --upgrade pip && pip install --progress-bar off  -r requirements.txt && ansible-galaxy collection install -r requirements.yml --collections-path "/usr/share/ansible/collections" && rm -rf /root/.ansible /root/.cache /root/.pip' returned a non-zero code: 1

kurokobo · April 9, 2024, 1:14pm

Still can’t reproduce your issue on 19.03.13.
As I mentioned, I would like to recommend you to upgrade your Docker

$ docker version
Client: Docker Engine - Community
 Version:           19.03.13
 API version:       1.40
 Go version:        go1.13.15
 Git commit:        4484c46d9d
 Built:             Wed Sep 16 17:02:36 2020
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.13
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       4484c46d9d
  Built:            Wed Sep 16 17:01:11 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.28
  GitCommit:        ae07eda36dd25f8a1b98dfbf587313b99c0190bb
 runc:
  Version:          1.1.12
  GitCommit:        v1.1.12-0-g51d5e94
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

$ tail -n+1 Dockerfile requirements.txt requirements.yml 
==> Dockerfile <==
FROM quay.io/ansible/awx-ee:23.2.0
USER root
COPY requirements.yml requirements.yml
COPY requirements.txt requirements.txt
RUN python3 -m pip install --progress-bar off --upgrade pip && pip install --progress-bar off -r requirements.txt && ansible-galaxy collection install -r requirements.yml && rm -rf /root/.ansible /root/.cache /root/.pip
USER 1000

==> requirements.txt <==
pandas
requests

==> requirements.yml <==
collections:

- name: awx.awx
- name: community.zabbix
- name: google.cloud
- name: community.general
- name: ansible.posix
- name: ansible.windows

$ docker build .
Sending build context to Docker daemon  4.096kB
Step 1/6 : FROM quay.io/ansible/awx-ee:23.2.0
23.2.0: Pulling from ansible/awx-ee
11e80c8b852f: Pull complete 
711e40327943: Pull complete 
ae79b81e22b0: Pull complete 
8bbe0b70a5f1: Pull complete 
62862f9df3e5: Pull complete 
9bca31556fb0: Pull complete 
23bb5e0a1426: Pull complete 
47d06bab5467: Pull complete 
f0d55f5ad39c: Pull complete 
08373c336785: Pull complete 
5b621dd61ba0: Pull complete 
4395bfc32c75: Pull complete 
4f4fb700ef54: Pull complete 
56dfeee262b5: Pull complete 
9250f6b083b3: Pull complete 
4e583f12d4ff: Pull complete 
4902e9eeeb15: Pull complete 
Digest: sha256:3695f4c0d11557da70fe8182ffe9046d5ce1df0770137f08f51705ab6c08ae05
Status: Downloaded newer image for quay.io/ansible/awx-ee:23.2.0
 ---> 277e9aa01faf
Step 2/6 : USER root
 ---> Running in 1b05a03d588d
Removing intermediate container 1b05a03d588d
 ---> bd3f3ea81cc3
Step 3/6 : COPY requirements.yml requirements.yml
 ---> 72b135f5b28e
Step 4/6 : COPY requirements.txt requirements.txt
 ---> 33512da9c000
Step 5/6 : RUN python3 -m pip install --progress-bar off --upgrade pip && pip install --progress-bar off -r requirements.txt && ansible-galaxy collection install -r requirements.yml && rm -rf /root/.ansible /root/.cache /root/.pip
 ---> Running in fa9a88eef536
Requirement already satisfied: pip in /usr/local/lib/python3.9/site-packages (23.2.1)
Collecting pip
  Obtaining dependency information for pip from https://files.pythonhosted.org/packages/8a/6a/19e9fe04fca059ccf770861c7d5721ab4c2aebc539889e97c7977528a53b/pip-24.0-py3-none-any.whl.metadata
  Downloading pip-24.0-py3-none-any.whl.metadata (3.6 kB)
Downloading pip-24.0-py3-none-any.whl (2.1 MB)
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 23.2.1
    Uninstalling pip-23.2.1:
      Successfully uninstalled pip-23.2.1
Successfully installed pip-24.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
Collecting pandas (from -r requirements.txt (line 1))
  Downloading pandas-2.2.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (19 kB)
Requirement already satisfied: requests in /usr/local/lib/python3.9/site-packages (from -r requirements.txt (line 2)) (2.31.0)
Collecting numpy<2,>=1.22.4 (from pandas->-r requirements.txt (line 1))
  Downloading numpy-1.26.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (61 kB)
Collecting python-dateutil>=2.8.2 (from pandas->-r requirements.txt (line 1))
  Downloading python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB)
Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.9/site-packages (from pandas->-r requirements.txt (line 1)) (2023.3.post1)
Collecting tzdata>=2022.7 (from pandas->-r requirements.txt (line 1))
  Downloading tzdata-2024.1-py2.py3-none-any.whl.metadata (1.4 kB)
Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib64/python3.9/site-packages (from requests->-r requirements.txt (line 2)) (3.2.0)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.9/site-packages (from requests->-r requirements.txt (line 2)) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.9/site-packages (from requests->-r requirements.txt (line 2)) (1.26.16)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.9/site-packages (from requests->-r requirements.txt (line 2)) (2023.7.22)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.9/site-packages (from python-dateutil>=2.8.2->pandas->-r requirements.txt (line 1)) (1.16.0)
Downloading pandas-2.2.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (13.0 MB)
Downloading numpy-1.26.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (18.2 MB)
Downloading python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB)
Downloading tzdata-2024.1-py2.py3-none-any.whl (345 kB)
Installing collected packages: tzdata, python-dateutil, numpy, pandas
  Attempting uninstall: python-dateutil
    Found existing installation: python-dateutil 2.8.1
    Uninstalling python-dateutil-2.8.1:
      Successfully uninstalled python-dateutil-2.8.1
Successfully installed numpy-1.26.4 pandas-2.2.1 python-dateutil-2.9.0.post0 tzdata-2024.1
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
[WARNING]: Collection redhatinsights.insights does not support Ansible version
2.15.4
Starting galaxy collection install process
Process install dependency map
Starting collection install process
'awx.awx:23.1.0' is already installed, skipping.
'google.cloud:1.2.0' is already installed, skipping.
'ansible.posix:1.5.4' is already installed, skipping.
'ansible.windows:2.1.0' is already installed, skipping.
Downloading https://galaxy.ansible.com/api/v3/plugin/ansible/content/published/collections/artifacts/community-zabbix-2.3.1.tar.gz to /root/.ansible/tmp/ansible-local-127vqdfhf1/tmp2_n3w6qj/community-zabbix-2.3.1-zxnjhzqx
Installing 'community.zabbix:2.3.1' to '/root/.ansible/collections/ansible_collections/community/zabbix'
community.zabbix:2.3.1 was installed successfully
Downloading https://galaxy.ansible.com/api/v3/plugin/ansible/content/published/collections/artifacts/ansible-netcommon-6.0.0.tar.gz to /root/.ansible/tmp/ansible-local-127vqdfhf1/tmp2_n3w6qj/ansible-netcommon-6.0.0-aoqgjv5h
Installing 'ansible.netcommon:6.0.0' to '/root/.ansible/collections/ansible_collections/ansible/netcommon'
ansible.netcommon:6.0.0 was installed successfully
Downloading https://galaxy.ansible.com/api/v3/plugin/ansible/content/published/collections/artifacts/ansible-utils-4.0.0.tar.gz to /root/.ansible/tmp/ansible-local-127vqdfhf1/tmp2_n3w6qj/ansible-utils-4.0.0-cl28bxhu
Installing 'ansible.utils:4.0.0' to '/root/.ansible/collections/ansible_collections/ansible/utils'
ansible.utils:4.0.0 was installed successfully
Downloading https://galaxy.ansible.com/api/v3/plugin/ansible/content/published/collections/artifacts/community-postgresql-3.4.0.tar.gz to /root/.ansible/tmp/ansible-local-127vqdfhf1/tmp2_n3w6qj/community-postgresql-3.4.0-vte99j7u
Installing 'community.postgresql:3.4.0' to '/root/.ansible/collections/ansible_collections/community/postgresql'
community.postgresql:3.4.0 was installed successfully
Downloading https://galaxy.ansible.com/api/v3/plugin/ansible/content/published/collections/artifacts/community-mysql-3.9.0.tar.gz to /root/.ansible/tmp/ansible-local-127vqdfhf1/tmp2_n3w6qj/community-mysql-3.9.0-4oa2m_00
Installing 'community.mysql:3.9.0' to '/root/.ansible/collections/ansible_collections/community/mysql'
community.mysql:3.9.0 was installed successfully
Downloading https://galaxy.ansible.com/api/v3/plugin/ansible/content/published/collections/artifacts/community-general-8.5.0.tar.gz to /root/.ansible/tmp/ansible-local-127vqdfhf1/tmp2_n3w6qj/community-general-8.5.0-4s7ftut2
Installing 'community.general:8.5.0' to '/root/.ansible/collections/ansible_collections/community/general'
community.general:8.5.0 was installed successfully
Removing intermediate container fa9a88eef536
 ---> 882240f4fbd5
Step 6/6 : USER 1000
 ---> Running in 7d84f484d68e
Removing intermediate container 7d84f484d68e
 ---> 13a4c3878ac2
Successfully built 13a4c3878ac2

SaravanaSelvaraj · April 10, 2024, 6:21am

Hello @kurokobo , No worries at all and Thank you so much for your help. We have figured out a workaround by building the image in local PC.

Have a great day ahead and Thanks again !

Best regards
Saravana Selvaraj

system · May 10, 2024, 6:21am

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.