Hey All,
I’ve been trying to deploy awx via ECS Fargate recently and am running into deployment issues. What I’ve done is build images based on Dockerfiles for:
-
awx_web
-
awx_task
-
memcache
-
awx_rabbitmq
that pull the ‘original’ awx images from dockerhub and inject kms encryption environment variables to the containers, which are decrypted with the aws-cli (installed via the dockerfile). These environment variables are given to the underlying container in the launch_awx.sh and launch_awx_task.sh files for their respective containers, and supervisord has access to them. The SECRET_KEY file is also created and populated with the SECRET_KEY environment variable from the injected kms secrets, then environment.sh is populated. None of this is done with installer or install.yml playbooks, all containers are deployed individually and the rabbitmq/awx_web/memcache/awx_task containers are fed environment variables that point to the ports / db info that they need.
Currently im stuck with this error:
`
2020-06-10 17:47:58,878 ERROR awx.conf.settings Database settings are not available, using defaults.
Traceback (most recent call last):
File “/var/lib/awx/venv/awx/lib/python3.6/site-packages/django/db/backends/base/base.py”, line 217, in ensure_connection
self.connect()
File “/var/lib/awx/venv/awx/lib/python3.6/site-packages/django/db/backends/base/base.py”, line 195, in connect
self.connection = self.get_new_connection(conn_params)
File “/var/lib/awx/venv/awx/lib/python3.6/site-packages/django/db/backends/postgresql/base.py”, line 178, in get_new_connection
connection = Database.connect(**conn_params)
File “/var/lib/awx/venv/awx/lib/python3.6/site-packages/psycopg2/init.py”, line 126, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
psycopg2.OperationalError: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket “/var/run/postgresql/.s.PGSQL.5432”?
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File “/var/lib/awx/venv/awx/lib/python3.6/site-packages/awx/conf/settings.py”, line 76, in _ctit_db_wrapper
yield
File “/var/lib/awx/venv/awx/lib/python3.6/site-packages/awx/conf/settings.py”, line 404, in getattr
value = self._get_local(name)
File “/var/lib/awx/venv/awx/lib/python3.6/site-packages/awx/conf/settings.py”, line 347, in _get_local
setting = Setting.objects.filter(key=name, user__isnull=True).order_by(‘pk’).first()
File “/var/lib/awx/venv/awx/lib/python3.6/site-packages/django/db/models/query.py”, line 653, in first
for obj in (self if self.ordered else self.order_by(‘pk’))[:1]:
File “/var/lib/awx/venv/awx/lib/python3.6/site-packages/django/db/models/query.py”, line 274, in iter
self._fetch_all()
File “/var/lib/awx/venv/awx/lib/python3.6/site-packages/django/db/models/query.py”, line 1242, in _fetch_all
self._result_cache = list(self._iterable_class(self))
File “/var/lib/awx/venv/awx/lib/python3.6/site-packages/django/db/models/query.py”, line 55, in iter
results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
File “/var/lib/awx/venv/awx/lib/python3.6/site-packages/django/db/models/sql/compiler.py”, line 1138, in execute_sql
cursor = self.connection.cursor()
File “/var/lib/awx/venv/awx/lib/python3.6/site-packages/django/db/backends/base/base.py”, line 256, in cursor
return self._cursor()
File “/var/lib/awx/venv/awx/lib/python3.6/site-packages/django/db/backends/base/base.py”, line 233, in _cursor
self.ensure_connection()
File “/var/lib/awx/venv/awx/lib/python3.6/site-packages/django/db/backends/base/base.py”, line 217, in ensure_connection
self.connect()
File “/var/lib/awx/venv/awx/lib/python3.6/site-packages/django/db/utils.py”, line 89, in exit
raise dj_exc_value.with_traceback(traceback) from exc_value
File “/var/lib/awx/venv/awx/lib/python3.6/site-packages/django/db/backends/base/base.py”, line 217, in ensure_connection
self.connect()
File “/var/lib/awx/venv/awx/lib/python3.6/site-packages/django/db/backends/base/base.py”, line 195, in connect
self.connection = self.get_new_connection(conn_params)
File “/var/lib/awx/venv/awx/lib/python3.6/site-packages/django/db/backends/postgresql/base.py”, line 178, in get_new_connection
connection = Database.connect(**conn_params)
File “/var/lib/awx/venv/awx/lib/python3.6/site-packages/psycopg2/init.py”, line 126, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
django.db.utils.OperationalError: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket “/var/run/postgresql/.s.PGSQL.5432”?
`
which has to deal with awx_web and awx_task not connecting to the postgresql db, however all of my environment variables are correct and I can see the db running both in rds and when i test locally. Anyone have an idea of what could be going wrong? Not sure why awx.conf.settings says the database settings arent available.
Environment variables that I inject are:
AWX_ADMIN_PASSWORD
AWX_ADMIN_USER
DATABASE_HOST
DATABASE_NAME
DATABASE_PASSWORD
DATABASE_PORT
DATABASE_USER
MEMCACHED_HOST
MEMCACHED_PORT
RABBITMQ_HOST
RABBITMQ_PASSWORD
RABBITMQ_PORT
RABBITMQ_USER
RABBITMQ_VHOST
SECRET_KEY