Rolloing out awx behind a proxy

Hello,

I am currently trying to roll out a docker test installation of awx behind a proxy. So for the installation to finish stage 10/34 should finish. It can not connect to the internet, thanks to the proxy we are using. It will run into a timeout.

 => ERROR [stage-1 10/34] RUN dnf install -y epel-next-release && dnf install -y inotify-tools && dnf remove -y epel-next-release                                                                         360.4s
------
 > [stage-1 10/34] RUN dnf install -y epel-next-release && dnf install -y inotify-tools && dnf remove -y epel-next-release:
0.341 Last metadata expiration check: 1:17:12 ago on Fri 12 Jan 2024 11:45:10 AM UTC.
0.405 Dependencies resolved.
0.406 ================================================================================
0.406  Package                 Architecture Version         Repository           Size
0.406 ================================================================================
0.406 Installing:
0.406  epel-next-release       noarch       9-7.el9         extras-common       8.1 k
0.406 Installing dependencies:
0.406  epel-release            noarch       9-7.el9         extras-common        19 k
0.406
0.406 Transaction Summary
0.406 ================================================================================
0.406 Install  2 Packages
0.406
0.406 Total download size: 27 k
0.406 Installed size: 29 k
0.407 Downloading Packages:
360.4
360.4 The downloaded packages were saved in cache until the next successful transaction.
360.4 You can remove cached packages by executing 'dnf clean packages'.
360.4 Error: Error downloading packages:
360.4   Curl error (28): Timeout was reached for https://mirrors.centos.org/metalink?repo=centos-extras-sig-extras-common-9-stream&arch=x86_64&protocol=https,http [Connection timed out after 30000 milliseconds]
------
Dockerfile.dev:166
--------------------
 164 |     ENV http_proxy http://xxx.xxx:xxx
 165 |     ENV https_proxy http://xxx.xxx:xxx
 166 | >>> RUN dnf install -y epel-next-release && dnf install -y inotify-tools && dnf remove -y epel-next-release
 167 |
 168 |     # Copy app from builder
--------------------
ERROR: failed to solve: process "/bin/sh -c dnf install -y epel-next-release && dnf install -y inotify-tools && dnf remove -y epel-next-release" did not complete successfully: exit code: 1
make: *** [Makefile:582: docker-compose-build] Fehler 1

A little bit of more information from my side. We are currently running Debian 12. Following modules are installed:

/awx$ docker -v
Docker version 24.0.7, build afdd53b

/awx$ docker compose version
Docker Compose version v2.21.0

I was wondering, if you uys could help me out and point out, where to add the proxy.

Thanks

@Dinkelzoid You may want to check if docker service is configured with an override to use the proxy config for its outbound http traffic.

$ sudo systemctl edit docker

[Service]
Environment="HTTP_PROXY=http://proxy:8080"
Environment="HTTPS_PROXY=http://proxy:8080"
Environment="NO_PROXY=IP sources you want to exclude from using the proxy" 

then restart docker service.

@cyru8 I already have that configured unter /etc/systemd/docker.service.d/http-proxy.conf So far I have figured out, that the Makefile has to be edited.So far I also think, that the Dockerfile.j2 has to be edited, since there are some issues.

I have fixed everything so far. make docker-compose throws some errors. I am not sure, if this is a bug. In the makefile ansible-galaxy install cant connect to the internet and runs into a timeout. Sadly it is not using the proxy, that I have configured in the /etc/environment.

ansible-galaxy install --ignore-certs -r tools/docker-compose/ansible/requirements.yml;
Starting galaxy collection install process
Process install dependency map
[WARNING]: Skipping Galaxy server https://galaxy.ansible.com. Got an unexpected error when getting available versions of collection awx.awx: Unknown error when attempting to call Galaxy at
'https://galaxy.ansible.com/api/': <urlopen error [Errno 101] Das Netzwerk ist nicht erreichbar>
ERROR! Unknown error when attempting to call Galaxy at 'https://galaxy.ansible.com/api/': <urlopen error [Errno 101] Das Netzwerk ist nicht erreichbar>
make: *** [Makefile:538: docker-compose] Fehler 1

It looks like I have to add a proxy somewhere in the files. Maybe somebody can point out, what i did wrong.

Okay I figured this one out. You have to add the proxy before ansible-galaxy install. Then it works like a charm. I am not sure, why my container starts and services in the container itself crash.

AttributeError: 'Settings' object has no attribute 'ANSIBLE_BASE_TEAM_MODEL'
2024-01-17 12:35:05,115 WARN exited: awx-rsyslog-configurer (exit status 1; not expected)
Traceback (most recent call last):
2024-01-17 12:35:05,149 INFO spawned: 'awx-rsyslog-configurer' with pid 888
  File "/usr/local/bin/awx-manage", line 9, in <module>
    load_entry_point('awx', 'console_scripts', 'awx-manage')()
  File "/awx_devel/awx/__init__.py", line 175, in manage
    execute_from_command_line(sys.argv)
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/core/management/__init__.py", line 416, in execute
    django.setup()
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/apps/registry.py", line 116, in populate
    app_config.import_models()
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/apps/config.py", line 269, in import_models
    self.models_module = import_module(models_module_name)
  File "/usr/lib64/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/ansible_base/models/__init__.py", line 3, in <module>
    from .organization import AbstractOrganization
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/ansible_base/models/organization.py", line 9, in <module>
    class AbstractOrganization(UniqueNamedCommonModel):
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/ansible_base/models/organization.py", line 28, in AbstractOrganization
    settings.ANSIBLE_BASE_TEAM_MODEL,
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/conf/__init__.py", line 104, in __getattr__
    val = getattr(_wrapped, name)
AttributeError: 'Settings' object has no attribute 'ANSIBLE_BASE_TEAM_MODEL'
useradd -g nginx nginx
useradd: user 'nginx' already exists
make[1]: Leaving directory '/awx_devel'
make[1]: *** [Makefile:249: nginx] Error 9
2024-01-17 12:35:05,283 WARN exited: awx-nginx (exit status 2; not expected)
2024-01-17 12:35:05,284 INFO spawned: 'awx-nginx' with pid 893
make[1]: Entering directory '/awx_devel'
Traceback (most recent call last):
  File "/usr/local/bin/awx-manage", line 9, in <module>
    load_entry_point('awx', 'console_scripts', 'awx-manage')()
  File "/awx_devel/awx/__init__.py", line 175, in manage
    execute_from_command_line(sys.argv)
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/core/management/__init__.py", line 416, in execute
    django.setup()
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/apps/registry.py", line 116, in populate
    app_config.import_models()
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/apps/config.py", line 269, in import_models
    self.models_module = import_module(models_module_name)
  File "/usr/lib64/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/ansible_base/models/__init__.py", line 3, in <module>
    from .organization import AbstractOrganization
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/ansible_base/models/organization.py", line 9, in <module>
    class AbstractOrganization(UniqueNamedCommonModel):
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/ansible_base/models/organization.py", line 28, in AbstractOrganization
    settings.ANSIBLE_BASE_TEAM_MODEL,
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/conf/__init__.py", line 104, in __getattr__
    val = getattr(_wrapped, name)
AttributeError: 'Settings' object has no attribute 'ANSIBLE_BASE_TEAM_MODEL'
2024-01-17 12:35:05,575 WARN exited: awx-ws-heartbeat (exit status 1; not expected)
2024-01-17 12:35:05,575 WARN exited: awx-wsrelay (exit status 1; not expected)
Traceback (most recent call last):
  File "/usr/local/bin/awx-manage", line 9, in <module>
2024-01-17 12:35:05,680 INFO spawned: 'awx-wsrelay' with pid 901
2024-01-17 12:35:05,682 INFO spawned: 'awx-ws-heartbeat' with pid 902
    load_entry_point('awx', 'console_scripts', 'awx-manage')()
  File "/awx_devel/awx/__init__.py", line 175, in manage
    execute_from_command_line(sys.argv)
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/core/management/__init__.py", line 416, in execute
    django.setup()
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/apps/registry.py", line 116, in populate
    app_config.import_models()
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/apps/config.py", line 269, in import_models
    self.models_module = import_module(models_module_name)
  File "/usr/lib64/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/ansible_base/models/__init__.py", line 3, in <module>
    from .organization import AbstractOrganization
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/ansible_base/models/organization.py", line 9, in <module>
    class AbstractOrganization(UniqueNamedCommonModel):
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/ansible_base/models/organization.py", line 28, in AbstractOrganization
    settings.ANSIBLE_BASE_TEAM_MODEL,
  File "/var/lib/awx/venv/awx/lib64/python3.9/site-packages/django/conf/__init__.py", line 104, in __getattr__
    val = getattr(_wrapped, name)
AttributeError: 'Settings' object has no attribute 'ANSIBLE_BASE_TEAM_MODEL'
2024-01-17 12:35:05,908 WARN exited: awx-dispatcher (exit status 1; not expected)
2024-01-17 12:35:05,908 WARN received SIGTERM indicating exit request
2024-01-17 12:35:05,908 INFO waiting for stdout, rsyslog-4xx-recovery, awx-receiver, awx-uwsgi, awx-daphne, awx-nginx, awx-wsrelay, awx-rsyslogd, awx-ws-heartbeat, awx-rsyslog-configurer, awx-cache-clear, awx-autoreload, awx-receptor to die
2024-01-17 12:35:05,909 WARN stopped: awx-receptor (terminated by SIGTERM)
make[1]: *** wait: No child processes.  Stop.
make[1]: *** Waiting for unfinished jobs....
make[1]: *** wait: No child processes.  Stop.
2024-01-17 12:35:05,910 WARN stopped: awx-autoreload (exit status 2)
2024-01-17 12:35:05,910 WARN stopped: awx-uwsgi (terminated by SIGKILL)
2024-01-17 12:35:05,910 WARN stopped: awx-nginx (terminated by SIGTERM)
2024-01-17 12:35:05,910 WARN stopped: awx-wsrelay (terminated by SIGTERM)
2024-01-17 12:35:05,911 INFO stopped: awx-rsyslogd (exit status 0)
2024-01-17 12:35:05,911 WARN stopped: awx-ws-heartbeat (terminated by SIGTERM)
2024-01-17 12:35:05,913 WARN stopped: awx-rsyslog-configurer (terminated by SIGTERM)
2024-01-17 12:35:05,913 WARN stopped: awx-receiver (terminated by SIGTERM)
2024-01-17 12:35:05,914 WARN stopped: awx-cache-clear (terminated by SIGTERM)
make[1]: *** [Makefile:229: daphne] Terminated
2024-01-17 12:35:05,914 WARN stopped: awx-daphne (terminated by SIGTERM)
2024-01-17 12:35:05,915 WARN stopped: rsyslog-4xx-recovery (terminated by SIGTERM)
2024-01-17 12:35:05,915 WARN stopped: stdout (terminated by SIGTERM)
make: *** wait: No child processes.  Stop.

This is the last traceback, that I got from the Docker container.

We are so glad that this has been solved! Would you mind marking this as “Solved” for us? Thank you for your time!