Build and use own Execution Environment - Failed with error

, ,

** Build and use own Execution Environment failed with error**

  • I have followed steps mentioned in awx-on-k3s/builder at main · kurokobo/awx-on-k3s · GitHub URL but got below error, Kindly help me to fix.
  • Planning to install AWX 24.6.1 with own Execution Environment
  • Server setup- RHEL 9, podman-docker-5.2.2-15.el9_5.noarch, Python 3.11 , ansible-builder
  • Error Logs
[root@server builder]# ansible-builder build --tag registry.example.com/ansible/ee:2.15-custom --container-runtime docker --verbosity 3
Ansible Builder is generating your execution environment build context.
File context/_build/requirements.yml will be created.
File context/_build/requirements.txt will be created.
File context/_build/bindep.txt will be created.
Creating context/_build/configs
File context/_build/configs/ansible.cfg will be created.
File context/_build/scripts/assemble will be created.
File context/_build/scripts/install-from-bindep will be created.
File context/_build/scripts/introspect.py will be created.
File context/_build/scripts/check_galaxy will be created.
File context/_build/scripts/check_ansible will be created.
File context/_build/scripts/pip_install will be created.
File context/_build/scripts/entrypoint will be created.
Ansible Builder is building your execution environment image. Tags: registry.example.com/ansible/ee:2.15-custom
Running command:
  docker build -f context/Dockerfile -t registry.example.com/ansible/ee:2.15-custom context
Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.
[1/4] STEP 1/17: FROM quay.io/centos/centos:stream9-minimal AS base
Trying to pull quay.io/centos/centos:stream9-minimal...
Getting image source signatures
Copying blob sha256:286eb63bc9616cc88e477f1292d3d27a5579b2964f05c29d0f5d03e0e1eb9c56
Copying config sha256:19387c59e3fe484ec2405c4e9bf897e45b2ed615b20f8f62a815b983a5f61004
Writing manifest to image destination
[1/4] STEP 2/17: USER root
--> d318a45dd812
[1/4] STEP 3/17: ENV PIP_BREAK_SYSTEM_PACKAGES=1
--> 538bf5ee95ed
[1/4] STEP 4/17: ARG EE_BASE_IMAGE
--> 1a28faac9058
[1/4] STEP 5/17: ARG PYCMD
--> 262e33395897
[1/4] STEP 6/17: ARG PYPKG
--> d582932d1542
[1/4] STEP 7/17: ARG PKGMGR_PRESERVE_CACHE
--> 7e64425417df
[1/4] STEP 8/17: ARG ANSIBLE_GALAXY_CLI_COLLECTION_OPTS
--> 2f475cf1f18c
[1/4] STEP 9/17: ARG ANSIBLE_GALAXY_CLI_ROLE_OPTS
--> 8b7e73d0d188
[1/4] STEP 10/17: ARG ANSIBLE_INSTALL_REFS
--> 57929d775767
[1/4] STEP 11/17: ARG PKGMGR
--> 9cbaa9faab11
[1/4] STEP 12/17: COPY _build/scripts/ /output/scripts/
--> 960f7d8b06c1
[1/4] STEP 13/17: COPY _build/scripts/entrypoint /opt/builder/bin/entrypoint
--> 7a080c15bbe6
[1/4] STEP 14/17: RUN $PKGMGR install $PYPKG -y ; if [ -z $PKGMGR_PRESERVE_CACHE ]; then $PKGMGR clean all; fi
Downloading metadata...

(microdnf:2): librepo-WARNING **: 11:01:40.889: LRO_METALINKURL processing failed: Curl error (60): SSL peer certificate or SSH remote key was not OK for https://mirrors.centos.org/metalink?repo=centos-extras-sig-extras-common-9-stream&arch=x86_64&protocol=https,http [SSL certificate problem: unable to get local issuer certificate]
error: cannot update repo 'extras-common': Cannot prepare internal mirrorlist: Curl error (60): SSL peer certificate or SSH remote key was not OK for https://mirrors.centos.org/metalink?repo=centos-extras-sig-extras-common-9-stream&arch=x86_64&protocol=https,http [SSL certificate problem: unable to get local issuer certificate]; Last error: Curl error (60): SSL peer certificate or SSH remote key was not OK for https://mirrors.centos.org/metalink?repo=centos-extras-sig-extras-common-9-stream&arch=x86_64&protocol=https,http [SSL certificate problem: unable to get local issuer certificate]
Complete.
--> 676e922b0abd
[1/4] STEP 15/17: RUN /output/scripts/pip_install $PYCMD
/usr/bin/python3.11 is not an executable
+ PYCMD=/usr/bin/python3.11
+ '[' -z /usr/bin/python3.11 ']'
+ '[' '!' -x /usr/bin/python3.11 ']'
+ echo '/usr/bin/python3.11 is not an executable'
+ exit 1
**Error: building at STEP "RUN /output/scripts/pip_install $PYCMD": while running runtime: exit status 1**

An error occurred (rc=1), see output line(s) above for details.
[root@server builder]
- my code!
Steps followed till Build EE and got error.

@kurokobo

I’ve seen this error when the base image an ee is built from doesn’t include python, but I can see that its being manually installed here.

The next thing I’d check is to see if python3.11 is actually the correct package name.

Is a proxy configured for Docker?

2 Likes

Echoing @kurokobo, it would appear something is man-in-the-middling your TLS sessions, likely a proxy of some kind if it’s an enterprise environment. It is possible to build an execution environment to include the root certificate authorities of all container images. A good stage is usually prepend_base; reference.

2 Likes

@kurokobo & @wayt good call there. I’d missed the

(microdnf:2): librepo-WARNING

bit in the middle.

And that probably prevents the image from installing the requested version of python, leading to final error about it not being executable.

1 Like

@kurokobo

Yes proxy is configured and i can see the proxy server details in “env” command in my server.
Also i able to connect proxy server.

You need to trust the root certificate in the container for your mitm proxy or do the bad thing and ignore failures which negates a lot of the benefit of encryption. In my post I link to the place in the execution-environment.yml to do that step at. We won’t know what root is, that’s going to be organization specific.

1 Like

Thank you @wayt

I have tried that option.

Logs and error

[root@server builder]# ansible-builder build --tag registry.example.com/ansible/ee:2.15-custom --container-runtime docker --verbosity 3
Ansible Builder is generating your execution environment build context.
File context/_build/requirements.yml is already up-to-date.
File context/_build/requirements.txt is already up-to-date.
File context/_build/bindep.txt is already up-to-date.
Creating context/_build/configs
File context/_build/configs/ansible.cfg is already up-to-date.
File context/_build/scripts/assemble is already up-to-date.
File context/_build/scripts/install-from-bindep is already up-to-date.
File context/_build/scripts/introspect.py is already up-to-date.
File context/_build/scripts/check_galaxy is already up-to-date.
File context/_build/scripts/check_ansible is already up-to-date.
File context/_build/scripts/pip_install is already up-to-date.
File context/_build/scripts/entrypoint is already up-to-date.
Ansible Builder is building your execution environment image. Tags: registry.example.com/ansible/ee:2.15-custom
Running command:
  docker build -f context/Dockerfile -t registry.example.com/ansible/ee:2.15-custom context
Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.
[1/4] STEP 1/19: FROM quay.io/centos/centos:stream9-minimal AS base
[1/4] STEP 2/19: USER root
--> Using cache d318a45dd812237c143f9c3013c0e79cc57a7745aa3385b4049421ed18b848d8
--> d318a45dd812
[1/4] STEP 3/19: ENV PIP_BREAK_SYSTEM_PACKAGES=1
--> Using cache 538bf5ee95edb30f467000da366d32af90024b2130750f81400fa3e208c58510
--> 538bf5ee95ed
[1/4] STEP 4/19: ARG EE_BASE_IMAGE
--> Using cache 1a28faac9058ade5dc2f4e009d332827df5ea107a31161cf7c515e907cea7fa6
--> 1a28faac9058
[1/4] STEP 5/19: ARG PYCMD
--> Using cache 262e33395897548812d09e235992cf872f350495b4a68b304abd64c1523285d7
--> 262e33395897
[1/4] STEP 6/19: ARG PYPKG
--> Using cache d582932d15421e73efe1f3c8f7c453d7dfc0463c808117f92fbb423e5fb66520
--> d582932d1542
[1/4] STEP 7/19: ARG PKGMGR_PRESERVE_CACHE
--> Using cache 7e64425417df6978f0a5d17e26d91b56a3819041c138311af7fe18a4c3794426
--> 7e64425417df
[1/4] STEP 8/19: ARG ANSIBLE_GALAXY_CLI_COLLECTION_OPTS
--> Using cache 2f475cf1f18c8060807d6f1375f5aebe03b1875ea913bb345e19a04b9687f659
--> 2f475cf1f18c
[1/4] STEP 9/19: ARG ANSIBLE_GALAXY_CLI_ROLE_OPTS
--> Using cache 8b7e73d0d18885e51bab0c5938bb2eb088634aa59204c4a774954c88d6210cba
--> 8b7e73d0d188
[1/4] STEP 10/19: ARG ANSIBLE_INSTALL_REFS
--> Using cache 57929d77576712163516bc521157e6f1b678c5da0fcb62f0f3e91d8fb13a3a16
--> 57929d775767
[1/4] STEP 11/19: ARG PKGMGR
--> Using cache 9cbaa9faab1178778c3043c4f1972d354bed11b87437863501eb950aeaffee14
--> 9cbaa9faab11
[1/4] STEP 12/19: COPY _build/scripts/ /output/scripts/
--> Using cache 960f7d8b06c179ef1450d8bc06bafb16ad20d610254caab4d0ed81b3c3e2444f
--> 960f7d8b06c1
[1/4] STEP 13/19: COPY _build/scripts/entrypoint /opt/builder/bin/entrypoint
--> Using cache 7a080c15bbe672bd6cf6641bcba8b2e615265e74c54d7b413cea32ed7e6e44cc
--> 7a080c15bbe6
[1/4] STEP 14/19: COPY ca-bundle.trust.crt /etc/path/ca-bundle.trust.crt
--> 7f351778e271
[1/4] STEP 15/19: RUN update-ca-trust
--> f7ceddd3893b
[1/4] STEP 16/19: RUN $PKGMGR install $PYPKG -y ; if [ -z $PKGMGR_PRESERVE_CACHE ]; then $PKGMGR clean all; fi
Downloading metadata...

(microdnf:2): librepo-WARNING **: 17:28:03.818: LRO_METALINKURL processing failed: Curl error (60): SSL peer certificate or SSH remote key was not OK for https://mirrors.centos.org/metalink?repo=centos-extras-sig-extras-common-9-stream&arch=x86_64&protocol=https,http [SSL certificate problem: unable to get local issuer certificate]
error: cannot update repo 'extras-common': Cannot prepare internal mirrorlist: Curl error (60): SSL peer certificate or SSH remote key was not OK for https://mirrors.centos.org/metalink?repo=centos-extras-sig-extras-common-9-stream&arch=x86_64&protocol=https,http [SSL certificate problem: unable to get local issuer certificate]; Last error: Curl error (60): SSL peer certificate or SSH remote key was not OK for https://mirrors.centos.org/metalink?repo=centos-extras-sig-extras-common-9-stream&arch=x86_64&protocol=https,http [SSL certificate problem: unable to get local issuer certificate]
Complete.
--> ce7acd585166
[1/4] STEP 17/19: RUN /output/scripts/pip_install $PYCMD
+ PYCMD=/usr/bin/python3.11
/usr/bin/python3.11 is not an executable
+ '[' -z /usr/bin/python3.11 ']'
+ '[' '!' -x /usr/bin/python3.11 ']'
+ echo '/usr/bin/python3.11 is not an executable'
+ exit 1
Error: building at STEP "RUN /output/scripts/pip_install $PYCMD": while running runtime: exit status 1

An error occurred (rc=1), see output line(s) above for details.
[root@server builder]#

Code

additional_build_steps:
  prepend_base:
    - COPY ca-bundle.trust.crt /etc/path/ca-bundle.trust.crt
    - RUN update-ca-trust

Not sure if that’s an obfuscation or if that might the issue /etc/path/ca-bundle.trust.crt. Usually you’ll want to add specific certificates (public keys) at a time and not a bundle and then run update-ca-trust which builds the appropriate bundles.

Let’s say you have a root certificate authority called VINO ROOT (in the CN). I’d add it as a file called VINOROOT.pem to the /etc/pki/ca-trust/source/anchors/ directory and then run update-ca-trust. I also wouldn’t try to copy an existing bundle over another system-wide certificates managed bundle.

You can do some troubleshooting, in your environment you likely will want to inspect the certificate chain using something like openssl s_client -connect mirrors.centos.org:443 -showcerts. You may need additional commands to ensure it goes through a proxy. It’s possible even with the right certificate that the chain itself isn’t properly offered by the proxy (the s_client tool helps find these). If that’s the case the fix is to either fix the proxy (sometimes harder than it sounds and outside the EE builders control) or add the necessary intermediate certificate(s) to the trust store too (bad practice, please don’t do this but …).

2 Likes

Thank you very much @wayt
By updating correct certification path- /etc/pki/ca-trust/source/anchors/., now i successfully able to run ansible-builder command and image built is completed, I will work on further setup.

[root@server builder]# docker image ls
Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.
REPOSITORY                       TAG              IMAGE ID      CREATED             SIZE
registry.example.com/ansible/ee  2.15-custom      6cb8cc953e07  36 seconds ago      299 MB
<none>                           <none>           180024c4a569  About a minute ago  301 MB
<none>                           <none>           ca9c4bd61984  2 minutes ago       251 MB
<none>                           <none>           ce7acd585166  20 hours ago        109 MB
<none>                           <none>           676e922b0abd  2 days ago          106 MB
quay.io/centos/centos            stream9-minimal  19387c59e3fe  8 days ago          106 MB
[root@server builder]#
1 Like