Need help with AWX Install using local Docker and external postgres database

Archives AWX Project
,
1

Hello,

I’d like to install AWX.
In an older version everything seemed to run as separate processes; however, I cloned down the latest version (looking in the VERSION file, I see 3.0.1), and it appears to support the following installations:

  1. openshift
  2. kubernetes
  3. docker or docker compose

I’d like to go with the 3rd option, specifically docker standalone and I’d like my postgres database to be on a different host where I have postgres installed and running.

This is how I set up my inventory file:

[all:vars]
dockerhub_base=ansible
dockerhub_version=latest
awx_task_hostname=awx
awx_web_hostname=awxweb
postgres_data_dir=/tmp/pgdocker
host_port=80
use_docker_compose=false
docker_registry=172.30.1.1:5000
docker_registry_repository=awx
docker_registry_username=root
docker_remove_local_images=False
pg_hostname=nameofremotehost
pg_username=awx
pg_password=hidden
pg_database=awx
pg_port=5432
rabbitmq_password=hidden
rabbitmq_erlang_cookie=hidden
admin_user=admin
admin_password=hidden
create_preload_data=True
secret_key=hidden
http_proxy=http://user:pass@ourproxy.domain.com:8080
http_proxys=https://user:pass@ourproxy.domain.com:8080
project_data_dir=/var/lib/awx/projects

When I run the playbook, I don’t see any failures… a number of tasks are skipped…

ansible-playbook -i inventory install.yml

PLAY [Build and deploy AWX] *****************************************************************************************************************************************************************************

TASK [check_vars : include_tasks] ***********************************************************************************************************************************************************************
skipping: [localhost]

TASK [check_vars : include_tasks] ***********************************************************************************************************************************************************************
included: /home/j8683/git/awx/installer/roles/check_vars/tasks/check_docker.yml for localhost

TASK [check_vars : postgres_data_dir should be defined] *************************************************************************************************************************************************
skipping: [localhost]

TASK [check_vars : host_port should be defined] *********************************************************************************************************************************************************
ok: [localhost] => {
“changed”: false,
“msg”: “All assertions passed”
}

TASK [image_build : Set global version if not provided] *************************************************************************************************************************************************
skipping: [localhost]

TASK [image_build : Verify awx-logos directory exists for official install] *****************************************************************************************************************************
skipping: [localhost]

TASK [image_build : Copy logos for inclusion in sdist] **************************************************************************************************************************************************
skipping: [localhost]

TASK [image_build : Set sdist file name] ****************************************************************************************************************************************************************
skipping: [localhost]

TASK [image_build : AWX Distribution] *******************************************************************************************************************************************************************
skipping: [localhost]

TASK [image_build : Stat distribution file] *************************************************************************************************************************************************************
skipping: [localhost]

TASK [image_build : Clean distribution] *****************************************************************************************************************************************************************
skipping: [localhost]

TASK [image_build : Build sdist builder image] **********************************************************************************************************************************************************
skipping: [localhost]

TASK [image_build : Build AWX distribution using container] *********************************************************************************************************************************************
skipping: [localhost]

TASK [image_build : Build AWX distribution locally] *****************************************************************************************************************************************************
skipping: [localhost]

TASK [image_build : Set docker build base path] *********************************************************************************************************************************************************
skipping: [localhost]

TASK [image_build : Set awx_web image name] *************************************************************************************************************************************************************
skipping: [localhost]

TASK [image_build : Set awx_task image name] ************************************************************************************************************************************************************
skipping: [localhost]

TASK [image_build : Ensure directory exists] ************************************************************************************************************************************************************
skipping: [localhost]

TASK [image_build : Stage sdist] ************************************************************************************************************************************************************************
skipping: [localhost]

TASK [image_build : Template web Dockerfile] ************************************************************************************************************************************************************
skipping: [localhost]

TASK [image_build : Template task Dockerfile] ***********************************************************************************************************************************************************
skipping: [localhost]

TASK [image_build : Stage launch_awx] *******************************************************************************************************************************************************************
skipping: [localhost]

TASK [image_build : Stage launch_awx_task] **************************************************************************************************************************************************************
skipping: [localhost]

TASK [image_build : Stage nginx.conf] *******************************************************************************************************************************************************************
skipping: [localhost]

TASK [image_build : Stage supervisor.conf] **************************************************************************************************************************************************************
skipping: [localhost]

TASK [image_build : Stage supervisor_task.conf] *********************************************************************************************************************************************************
skipping: [localhost]

TASK [image_build : Stage settings.py] ******************************************************************************************************************************************************************
skipping: [localhost]

TASK [image_build : Stage requirements] *****************************************************************************************************************************************************************
skipping: [localhost]

TASK [image_build : Stage config watcher] ***************************************************************************************************************************************************************
skipping: [localhost]

TASK [image_build : Stage Makefile] *********************************************************************************************************************************************************************
skipping: [localhost]

TASK [image_build : Stage ansible repo] *****************************************************************************************************************************************************************
skipping: [localhost]

TASK [image_build : Stage ansible repo key] *************************************************************************************************************************************************************
skipping: [localhost]

TASK [image_build : Build base web image] ***************************************************************************************************************************************************************
skipping: [localhost]

TASK [image_build : Build base task image] **************************************************************************************************************************************************************
skipping: [localhost]

TASK [image_build : Tag task and web images as latest] **************************************************************************************************************************************************
skipping: [localhost]

TASK [image_build : Clean docker base directory] ********************************************************************************************************************************************************
skipping: [localhost]

TASK [image_push : Authenticate with Docker registry if registry password given] ************************************************************************************************************************
skipping: [localhost]

TASK [image_push : Remove web image] ********************************************************************************************************************************************************************
skipping: [localhost]

TASK [image_push : Remove task image] *******************************************************************************************************************************************************************
skipping: [localhost]

TASK [image_push : Tag and push web image to registry] **************************************************************************************************************************************************
skipping: [localhost]

TASK [image_push : Tag and push task image to registry] *************************************************************************************************************************************************
skipping: [localhost]

TASK [image_push : Set full image path for Registry] ****************************************************************************************************************************************************
skipping: [localhost]

TASK [kubernetes : fail] ********************************************************************************************************************************************************************************
skipping: [localhost]

TASK [kubernetes : include_tasks] ***********************************************************************************************************************************************************************
skipping: [localhost] => (item=openshift_auth.yml)
skipping: [localhost] => (item=openshift.yml)

TASK [kubernetes : include_tasks] ***********************************************************************************************************************************************************************
skipping: [localhost] => (item=kubernetes_auth.yml)
skipping: [localhost] => (item=kubernetes.yml)

TASK [kubernetes : Use kubectl or oc] *******************************************************************************************************************************************************************
skipping: [localhost]

TASK [kubernetes : set_fact] ****************************************************************************************************************************************************************************
skipping: [localhost]

TASK [kubernetes : Record deployment size] **************************************************************************************************************************************************************
skipping: [localhost]

TASK [kubernetes : Set expected post-deployment Replicas value] *****************************************************************************************************************************************
skipping: [localhost]

TASK [kubernetes : Delete existing Deployment] **********************************************************************************************************************************************************
skipping: [localhost]

TASK [kubernetes : Get Postgres Service Detail] *********************************************************************************************************************************************************
skipping: [localhost]

TASK [kubernetes : Template PostgreSQL Deployment (OpenShift)] ******************************************************************************************************************************************
skipping: [localhost]

TASK [kubernetes : Deploy and Activate Postgres (OpenShift)] ********************************************************************************************************************************************
skipping: [localhost]

TASK [kubernetes : Deploy and Activate Postgres (Kubernetes)] *******************************************************************************************************************************************
skipping: [localhost]

TASK [kubernetes : Set postgresql hostname to helm package service (Kubernetes)] ************************************************************************************************************************
skipping: [localhost]

TASK [kubernetes : Wait for Postgres to activate] *******************************************************************************************************************************************************
skipping: [localhost]

TASK [kubernetes : Set task image name] *****************************************************************************************************************************************************************
skipping: [localhost]

TASK [kubernetes : Set web image name] ******************************************************************************************************************************************************************
skipping: [localhost]

TASK [kubernetes : Render deployment templates] *********************************************************************************************************************************************************
skipping: [localhost] => (item=None)
skipping: [localhost] => (item=None)
skipping: [localhost] => (item=None)

TASK [kubernetes : Apply Deployment] ********************************************************************************************************************************************************************
skipping: [localhost]

TASK [kubernetes : Delete any existing management pod] **************************************************************************************************************************************************
skipping: [localhost]

TASK [kubernetes : Template management pod] *************************************************************************************************************************************************************
skipping: [localhost]

TASK [kubernetes : Create management pod] ***************************************************************************************************************************************************************
skipping: [localhost]

TASK [kubernetes : Wait for management pod to start] ****************************************************************************************************************************************************
skipping: [localhost]

TASK [kubernetes : Migrate database] ********************************************************************************************************************************************************************
skipping: [localhost]

TASK [kubernetes : Check for Tower Super users] *********************************************************************************************************************************************************
skipping: [localhost]

TASK [kubernetes : create django super user if it does not exist] ***************************************************************************************************************************************
skipping: [localhost]

TASK [kubernetes : update django super user password] ***************************************************************************************************************************************************
skipping: [localhost]

TASK [kubernetes : Create the default organization if it is needed.] ************************************************************************************************************************************
skipping: [localhost]

TASK [kubernetes : Delete management pod] ***************************************************************************************************************************************************************
skipping: [localhost]

TASK [kubernetes : Scale up deployment] *****************************************************************************************************************************************************************
skipping: [localhost]

TASK [local_docker : Export Docker web image if it isnt local and there isnt a registry defined] ********************************************************************************************************
skipping: [localhost]

TASK [local_docker : Export Docker task image if it isnt local and there isnt a registry defined] *******************************************************************************************************
skipping: [localhost]

TASK [local_docker : Set docker base path] **************************************************************************************************************************************************************
skipping: [localhost]

TASK [local_docker : Ensure directory exists] ***********************************************************************************************************************************************************
skipping: [localhost]

TASK [local_docker : Copy web image to docker execution] ************************************************************************************************************************************************
skipping: [localhost]

TASK [local_docker : Copy task image to docker execution] ***********************************************************************************************************************************************
skipping: [localhost]

TASK [local_docker : Load web image] ********************************************************************************************************************************************************************
skipping: [localhost]

TASK [local_docker : Load task image] *******************************************************************************************************************************************************************
skipping: [localhost]

TASK [local_docker : Set full image path for local install] *********************************************************************************************************************************************
skipping: [localhost]

TASK [local_docker : Set DockerHub Image Paths] *********************************************************************************************************************************************************
ok: [localhost]

TASK [local_docker : Activate postgres container] *******************************************************************************************************************************************************
skipping: [localhost]

TASK [local_docker : Activate rabbitmq container] *******************************************************************************************************************************************************
ok: [localhost]

TASK [local_docker : Activate memcached container] ******************************************************************************************************************************************************
ok: [localhost]

TASK [local_docker : Wait for postgres and rabbitmq to activate] ****************************************************************************************************************************************
skipping: [localhost]

TASK [local_docker : Set properties without postgres for awx_web] ***************************************************************************************************************************************
ok: [localhost]

TASK [local_docker : Set properties with postgres for awx_web] ******************************************************************************************************************************************
skipping: [localhost]

TASK [local_docker : Set properties without postgres for awx_task] **************************************************************************************************************************************
ok: [localhost]

TASK [local_docker : Set properties with postgres for awx_task] *****************************************************************************************************************************************
skipping: [localhost]

TASK [local_docker : Activate AWX Web Container] ********************************************************************************************************************************************************
changed: [localhost]

TASK [local_docker : Update CA trust in awx_web container] **********************************************************************************************************************************************
changed: [localhost]

TASK [local_docker : Activate AWX Task Container] *******************************************************************************************************************************************************
changed: [localhost]

TASK [local_docker : Update CA trust in awx_task container] *********************************************************************************************************************************************
changed: [localhost]

TASK [local_docker : Create {{ docker_compose_dir }} directory] *****************************************************************************************************************************************
skipping: [localhost]

TASK [local_docker : Create docker-compose.yml file] ****************************************************************************************************************************************************
skipping: [localhost]

TASK [local_docker : Start the containers] **************************************************************************************************************************************************************
skipping: [localhost]

TASK [local_docker : Update CA trust in awx_web container] **********************************************************************************************************************************************
skipping: [localhost]

TASK [local_docker : Update CA trust in awx_task container] *********************************************************************************************************************************************
skipping: [localhost]

PLAY RECAP **********************************************************************************************************************************************************************************************
localhost : ok=11 changed=4 unreachable=0 failed=0

I see the following containers:

docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
cbcd749cc4fd ansible/awx_task:latest “/tini – /bin/sh …” 12 minutes ago Up 12 minutes 8052/tcp awx_task
01886fa5b0c3 ansible/awx_web:latest “/tini – /bin/sh …” 12 minutes ago Up 12 minutes 0.0.0.0:80->8052/tcp awx_web
af327e257f3a memcached:alpine “docker-entrypoint…” 6 hours ago Up 21 minutes 11211/tcp memcached
accad778e21c ansible/awx_rabbitmq:3.7.4 “docker-entrypoint…” 6 hours ago Up 21 minutes 4369/tcp, 5671-5672/tcp, 15671-15672/tcp, 25672/tcp rabbitmq

However, when I hit the host ip, I see the following:

Server Error
A server error has occurred.

I don’t see the database created on my external host, so that’s my first guess as the cause of my issue… I need to get the database installed there; however, I thought my pg_hostname variable being set was going to take care of that.

Do you see what I’m missing?

Thanks,

Radesh

2

That was wrong. Setting up PG details in the inventory file helps to point the DB details in awx_web and awx_task container.

I would recommend to enable use_docker_compose option and try to grasp an idea how all these containers are linked each other. Then you can separate postgres and build it on remote host and connect everything together via docker-compose

3

Sujith,

Thank you for your response.
I don’t want to have my DB running as a container.

Taking a step back, is it possible to install awx in a manner where the playbook will install my other components (awx_web, etc) as containers, and deploy the database to another server?

Thanks,

Radesh

4

To summarize: you want to deploy AWX using an external database and

I don’t see the database created on my external host, so that’s my first guess as the cause of my issue… I need to get the database installed there; however, I thought my pg_hostname variable being set was going to take care of that.

IIRC AWX doesn’t provision the external database for you. You need to install postgres, configure postgres, create the awx database, create the awx user, and use those connection details in the inventory file. Also, look at your awx_web and awx_task container logs since they usually have enough information to figure out what the issue is, this time I’m guessing the container couldn’t find a listening postgres server.

5

Uriel,

Thank you for clarifying. I should have stated that I have postgres installed and running on another host.
I misunderstood the instructions and didn’t realize that I would need to create the role and the database on the postgres server prior to running the installation playbook.

I’ve gone back to the postgres server and created a role with the password specified in my inventory file, and a database called awx…

awx=# \l
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges

6

It should. If you navigate to the AWX homepage and still see server errors, check the logs on the awx_web container. Good luck!

7

I ran the install again, but still don’t have it up and running.
I’ve got some information to confirm that the issue is with the connection to the database.

This is what is running…

On my host where I’ve got containers for awx_task, awx_web, memcached, and rabbitmq…

[root@lnx21667 j8683]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
0a0d0422e4f3 ansible/awx_task:latest “/tini – /bin/sh …” 2 hours ago Up 2 hours 8052/tcp awx_task
6b7d70678c09 ansible/awx_web:latest “/tini – /bin/sh …” 2 hours ago Up 2 hours 0.0.0.0:80->8052/tcp awx_web
e559e1342d49 memcached:alpine “docker-entrypoint…” 2 hours ago Up 2 hours 11211/tcp memcached
356a78e5f4b5 ansible/awx_rabbitmq:3.7.4 “docker-entrypoint…” 2 hours ago Up 2 hours 4369/tcp, 5671-5672/tcp, 15671-15672/tcp, 25672/tcp rabbitmq
[root@lnx21667 j8683]#

On my host where I have postgres running:

-bash-4.2$ uname -n; whoami
lnx21666
postgres
-bash-4.2$ psql
psql (9.6.12)
Type “help” for help.

postgres=# \l
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges

(attachments)

install.san.log (104 KB)

8

I got it!
I noticed error messages complaining about DATABASE_SSLMODE=__omit_place_holder__2da4557f2219960d48f7e07b10b232b340c28493
I didn’t want to configure SSL on this DB… may want to down the road, but not on my initial setup, so I played with the options.
Setting pg_sslmode to disable then running the playbook again worked!

I’m sitting at the log in page, which is what I was expecting.
From here I should be good to go.

Next steps… HA this bad boy.

Radesh

9

In my excitement I forgot to say, “Thanks!” to Uriel and Sujith.
My bad :).

Radesh

10

(:

There’s been discussion around setting up AWX HA/clustering on local docker/Swarm but I don’t think that was ever completely figured out. AWX proper only supports HA/clustering on Kubernetes. But you should be able to set up HA on the postgres server.

11

The AWX HA / Clustering can be done via local docker method, however some tweaks needed . Check this thread for any future updates

https://groups.google.com/forum/m/?utm_medium=email&utm_source=footer#!msg/awx-project/E6mlcdfED2s/MR3dtETLBAAJ

12

I can’t remember which version of Tower I was running, but we had Red Hat in to help us get started with using Tower.
In our discussions with them they basically told us that making Tower HA meant adding more web frontends.
So, really that is my expectation for this… add more frontends.

That said, I’ve read some interesting stuff on this list related to clustering RabbitMQ, but… one step at a time.

13

That’s not a strictly true statement… it’s adding more web front-ends and more execution capacity.

14

Matthew,

Thank you for that clarification.
I’m certainly looking forward to exploring how we can HA it.

We’ve got our AWX stuff running on RHV, and our Tower implementation on VM (vmware guest). I’m looking forward to implementing in our OpenShift environment eventually.

Radesh