Create custom images, push them to google cloud registry and deploy awx on kubernetes

Archives AWX Project
,
1

Hi,

I want to create a custom version of awx_task (with dnspython and openshift installed) and push this image to gcr.

I have modified requirements/requirements_ansible.txt to add dnspython and openshift. The build is successful and I have awx_task modified as I want.

[james@e6230 awx-3.0.0]$ sudo docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
awx_task 3.0.0 5b7592c577c3 25 minutes ago 1.05 GB
awx_task latest 5b7592c577c3 25 minutes ago 1.05 GB
awx_web 3.0.0 c31bc1361abb 26 minutes ago 1.02 GB
awx_web latest c31bc1361abb 26 minutes ago 1.02 GB

As far as I read I can’t see how to achieve my goal.

The images are local and I don’t know how to push them on gcr and how to deploy awx with this images.

If someone can help me … it will be appreciated.

James

2

That question is really about deploying images to GCP, which seems outside the scope of this list. You can export a docker image to a tar file. Or tag it and push to a docker registry where you might be able to pull it within GCP.

Dave

3

Hi David,
Thanks for you answer. I know how to deploy image from private gcr to gke (on gcp). My question was about the inventory file.

I have set all the mandatory variables to be able to push and pull images from gcr but maybe I am missing something.

From the command line I can push and pull images to gcr.

inventory file looks like :

4

I see what you’re trying to do. Sorry, I didn’t get that from your initial question.

Are you setting registry authentication variables? Looking at the installer/roles/image_push/tasks/main.yml file, the first docker_login task oly happens when the docker_registry and docker_registry_password variables are defined. When you run the install.yml playbook, does it perform the tasks in the image_push role, or does it skip them?

Here’s what I have working. I’m using OpenShift Origin, but it is still just a docker registry.

docker_registry: docker-registry.default.svc:5000
docker_registry_repository: awx
docker_registry_username: unused
docker_registry_password: “{{ token }}”

I’m using an auth token instead of a username and password. I set the token variable when I run:

ansible-playbook install.yml -e token=$(oc whoami -t)

Your kubectl probably has a similar (or exactly the same) command.

Hope that helps,

Dave

5

I have follow your advice and find out how I can get a token to access gcr.

https://cloud.google.com/container-registry/docs/advanced-authentication#access_token

the inventory file looks like :

docker_registry=gcr.io
docker_registry_repository=mycontext-123456
docker_registry_username=oauth2accesstoken

and the command line is :

6

It does look like it’s working. You might try adding -vvv to the ansible-playbook command to see if it gives you anything useful.

7

some observations :

  1. the awx containers are built with the new modules (dnspython and openshift) and the images are on my laptop
(attachments)

awx-k8s.txt (200 KB)

8

I have found something very strange. To be able to create custom images, push them to GCR and run AWX from this images, I have to comment

#kubernetes_task_image: “{{ tower_package_name | default(‘ansible/awx_task’) }}”
#kubernetes_web_image: “{{ tower_package_name | default(‘ansible/awx_web’) }}”

in **installer/roles/kubernetes/defaults/main.yml** . If kubernetes_task_image and kubernetes_web_image are not commented, it's impossible to use the build images.

Now I have another question. Once AWX is deployed with the custom images and the custom modules (dnspython and openshift) is installed, I still cannot run my playbook. I've got this message :

Using /etc/ansible/ansible.cfg as config file
/tmp/awx_13_xxfdcqtw/tmpwqww2_5w did not meet host_list requirements, check plugin documentation if this is unexpected

PLAY [localhost] ***************************************************************

TASK [deploy-on-k8s : debug] ***************************************************
ok: [localhost] => {
"msg": "1.17"
}

TASK [deploy-on-k8s : Fetch all certificates in a Namespace] *******************
fatal: [localhost]: FAILED! => {"changed": false, "msg": "This module requires the OpenShift Python client. Try `pip install openshift`"}

PLAY RECAP *********************************************************************
localhost : ok=1 changed=0 unreachable=0 failed=1

Any help will be welcome.
9

I had to set :

ansible_connection: local
ansible_python_interpreter: “/var/lib/awx/venv/ansible/bin/python2”

in my inventory .

After that everything works as expected.