Ansible builder failing on new host

Get Help
, , , , , ,
1

After working on my dev environment I am starting to get things going in our prod. I need to build the EE from a jump host and deploy it to AAP. However I am running into issues.
This is the error I am getting from the ansible-builder job:

Installing collected packages: zipp, ptyprocess, lockfile, pyyaml, python-daemon, pexpect, packaging, importlib-metadata, ansible-runner
Successfully installed ansible-runner-2.4.2 importlib-metadata-6.2.1 lockfile-0.12.2 packaging-25.0 pexpect-4.9.0 ptyprocess-0.7.0 python-daemon-3.1.2 pyyaml-6.0.3 zipp-3.23.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
WARNING: You are using pip version 21.3.1; however, version 25.3 is available.
You should consider upgrading via the '/usr/bin/python3.9 -m pip install --upgrade pip' command.
--> 9c6b1dd076fe
[1/4] STEP 29/29: RUN alternatives --install /usr/bin/python3 python3 /usr/bin/python3.9 0
--> b72057092dd8
[4/4] STEP 1/23: FROM b72057092dd8bd0beee7ffe9a5dcb38ed0f0a6221fb07bf780b57581fa7043f3 AS final
[4/4] STEP 2/23: ENV PIP_BREAK_SYSTEM_PACKAGES=1
--> 05747d1eb3ac
[4/4] STEP 3/23: ARG EE_BASE_IMAGE
--> 0a944f1a97ea
[4/4] STEP 4/23: ARG PYCMD
--> 1d741b59abdc
[4/4] STEP 5/23: ARG PYPKG
--> b42e4e13d5fd
[4/4] STEP 6/23: ARG PKGMGR_PRESERVE_CACHE
--> 5ed6d12467fd
[4/4] STEP 7/23: ARG ANSIBLE_GALAXY_CLI_COLLECTION_OPTS
--> 9651ada4dc12
[4/4] STEP 8/23: ARG ANSIBLE_GALAXY_CLI_ROLE_OPTS
--> 90fe2e37aac3
[4/4] STEP 9/23: ARG ANSIBLE_INSTALL_REFS
--> 309f0db56d2c
[4/4] STEP 10/23: ARG PKGMGR
--> 7b367c824fa4
[4/4] STEP 11/23: RUN /output/scripts/check_ansible $PYCMD
+ PYCMD=/usr/bin/python3.9
+ '[' -z /usr/bin/python3.9 ']'
+ '[' '!' -x /usr/bin/python3.9 ']'
+ ansible --version
ansible [core 2.15.13]
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/home/runner/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3.11/site-packages/ansible
  ansible collection location = /home/runner/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/bin/ansible
  python version = 3.11.13 (main, Aug 21 2025, 00:00:00) [GCC 11.5.0 20240719 (Red Hat 11.5.0-11)] (/usr/bin/python3.11)
  jinja version = 3.1.6
  libyaml = True
+ '[' 0 -ne 0 ']'
+ ansible-runner --version
2.4.2
+ '[' 0 -ne 0 ']'
+ /usr/bin/python3.9 -c 'import ansible ; import ansible_runner'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'ansible'
+ '[' 1 -ne 0 ']'
+ cat
**********************************************************************
ERROR - Missing Ansible or Ansible Runner for selected Python

  An Ansible and/or Ansible Runner installation cannot be found in
  the final builder image using the following Python interpreter:

    /usr/bin/python3.9

  Ansible and Ansible Runner must be installed in the final image and
  available to the selected Python interpreter. If you are using a
  recent enough version of the execution environment file, you may use
  the 'dependencies.ansible_core' configuration option to install
  Ansible and the 'dependencies.ansible_runner' configuration option
  to install Ansible Runner. You can also use 'additional_build_steps'
  to manually do this yourself. Alternatively, use a base image with
  Ansible and Ansible Runner already installed.
**********************************************************************
+ exit 1
[2/4] STEP 1/16: FROM b72057092dd8bd0beee7ffe9a5dcb38ed0f0a6221fb07bf780b57581fa7043f3 AS galaxy
[2/4] STEP 2/16: ARG EE_BASE_IMAGE
Error: building at STEP "RUN /output/scripts/check_ansible $PYCMD": while running runtime: exit status 1

Here is my execution-environment.yml

---
version: 3

dependencies:
  galaxy: requirements.yml
  python: requirements.txt
  system: bindep.txt
  python_interpreter:
    package_system: python3.9
    python_path: /usr/bin/python3.9
  ansible_runner:
    package_pip: ansible-runner

options:
  package_manager_path: /usr/bin/microdnf

images:
  base_image:
    #    name: nexus.example.com:9093/ansible-automation-platform-25/ee-minimal-rhel9:latest
    name: registry.redhat.io/ansible-automation-platform-24/ee-minimal-rhel9:latest
      #name: 'nexus.example.com:9093/ubi9/python-311:latest'

additional_build_files:
  - src: files/ansible.cfg  # Path to your local ansible.cfg
    dest: configs

additional_build_steps:
  append_base:
    - RUN alternatives --install /usr/bin/python3 python3 /usr/bin/python3.9 0
  prepend_base: |
    ENV http_proxy=http://INT.IP.ADDR:8080
    ENV HTTP_PROXY=${http_proxy}
    ARG http_proxy
    ENV no_proxy="127.0.0.1,::1,.example.com,10.0.0.0/8,192.168.0.0/16,172.16.0.0/12"
    ARG no_proxy
    ENV PIP_INDEX_URL="http://nexus.example.com/repository/pypi-proxy/simple"
    ARG PIP_INDEX_URL
    ENV PIP_TRUSTED_HOST="nexus.example.com"
    ARG PIP_TRUSTED_HOST
    RUN pip3 config set global.index-url http://nexus.example.com/repository/pypi-proxy/simple
    RUN pip3 config set global.trusted-host nexus.example.com
    RUN pip3 install --upgrade pip
    #    RUN pip3 install pyvmomi
  prepend_galaxy:
    - COPY _build/configs/ansible.cfg /etc/ansible/ansible.cfg

requirements.yml

---

collections:

  - name: community.general
  - name: community.vmware
  - name: vmware.vmware_rest
  - name: vmware.vmware
  - name: theforeman.foreman
  - name: venafi.machine_identity
  - name: cyberark.conjur
  - name: cyberark.pas
  - name: community.postgresql
  - name: containers.podman
  - name: ansible.controller
  - name: awx.awx
  - name: ansible.utils
  - name: community.aws

requirements.txt

pyvmomi
pypsrp
requests
setuptools
pyyaml
jinja2
jsonschema
jmespath
cryptography
pyopenssl