We have Gitlab, and I'm curious how other people are using it with AWX/AAP, Molecule, and CICD

Get Help
, , , ,
1

We are using Gitlab and we’ve done some pretty interesting things with it.

There isn’t a lot of best-practice, or design ideas out on the web for integrating the two. So I wanted to see what other people have gotten up to with it. I’d like to spark some conversation about successes/failures.

Thus far, we have had done some work with

  1. Getting repositories to automatically build execution environments
  2. Triggering AWX jobs via awxkit and awx.awx modules
  3. Updating projects after commits or merges
  4. Creating sets of templates that run against a branch
  5. Running ansible directly from execution environments pulled via image:

If you’re reading this, and you’re thinking “Wow, that’s pretty basic”. You’re exactly who I’d like to hear from. We are just starting with this, and I’m curious about the problems we are going to have in 6 months, with how we’re doing things now. You probably already know.

When I first started looking for a way to build an ee with a runner, it took me a week to find anything. It was a link to some repo on a private gitlab instance that was shared publicly. The method he was using was a little cumbersome, but I think it was built before a lot of the things were available. Anyway, we used it to reverse engineer a method of our own.

I’m really interested in Molecule tests. We have dabbled bit it, but our production environments are extremely legacy, most examples are using containers for the tests, we have fairly even split of Windows/Linux, in most cases both are required for any functional testing, and we would love to find a way to at least start moving the ball forward.

We’ve also been wondering about the nature of collection testing vs. role testing. If I commit a change to a role, it should test that role. However, that role might be a dependency, so one would think that the entire collection should be tested before the merge is permitted.

4 Likes
2

Hy i am also using Gitlab to automate the awx the only difference that i do is that i am using webhooks to tirgger a job on awx.

Like example if i commit to the main branch of my repo the awx will pull the latest changes and execute the playbook/role i have setup this like for the provisioning of the servers.

like adding / updating users , management of firewall deployment of ssl certificates…

3

I’m not using AWX/AAP or Windows but I am running Molecule via GitLab CI for some Ansible roles to test them on Debian and Ubuntu.

Getting systemd running in Docker containers via GitLab CI took a while and I have many roles that I haven’t yet setup Molecule for, the more notable ones that do have Molecule tests are:

I hope to add Molecule test for more roles, one issue I’ve yet to work out how to address is testing roles that depend on other services like Let’s Encrypt certs being provisioned (Apache, Nginx) or DNS entries (DKIM for Exim).

Also for old and complicated roles like the user role the number of ways it can use used (eg setting up users quotas which might require seperate /home partition and for them to be remounted, or configuring a server to provide multiple WordPress sites with Matomo stats or configuring a Debian server to run Nextcloud and ONLYOFFICE) it’s hard to see how I’ll find time to work out how write a comprehensive set of tests — it is actually easier to run a development server to test things like this against — I do have a repo that is used to build dev servers for testing.

For new roles I’m creating, like this Unbound one I’m now writing the Molecule config at the same time as the role rather than adding it on afterwards.

2 Likes
4

Hello. I’m interested in exactly the same! :bulb: :thought_balloon:
Thanks for asking & sharing! Imo, you’ve done quite well. :clap:t2:

We used the two pretty intensively for nearly 4 years and only recently we did a CI/CD tool switch here, however that doesn’t change anything. What we did found is that most of use cases involves job or workflow trigger through awxkit and awx.awx or now as we moved on to AAP via ansible.controller modules.

Thats how we mostly think about the two, so we can leverage the strong side of each tool and use it that way. Therefore to run most of operational task, like build ee or to onboard new AWX/AAP organization along with all defaults or custom teams, create inventories, assign roles etc, update mappings… we use ansible itself and leverage hooks to launch it.

However for many day two ops we rarely want in the long run the initial trigger to be a direct commit by a person. A jinja template and robot user can manage quite well. Whenever we can we hook it up to a nice UI form with sanitised inputs, safe defaults and we decouple ourselves from the process as we do provide a bunch of tools internally for product teams to use.

2 Likes
5

Our AWX controller isn’t able to receive a webhook directly from Gitlab, at least for the moment. I’m actually comfortable with that right now. It’s easy enough to build a job that uses a running to kick off the job.

Although you got me thinking about using curl from a running to possibly kick things off as well. Sometimes there are far too many options. :slightly_smiling_face:

6

We are getting close to being able to use vmware instant clones as ephemerals. My hope is that this might be able to be used for Molecule as opposed to a container. They are COW, they boot active, and take far less time to create than a regular VM clone.

I have actually built a structure that allows me to clone our production environments (one for each client) into an isolated zone without modify it. We can then test against that. However, it is much more time consuming, given that some clients have up to 7 vms. So I try to bring one in and keep it there, then use snapshots for rolling it back to a starting point. We’ve had a lot of success with this.

1 Like
7

You are just fine with GL launching job templates or workflows using awxkit. Its convenient. Using curl or any other way to call api directly is great when there are no other options.

e.g. we used to launch workflow job templates directly from Jira using groovy script added as Post Function to a specific status transition. Using custom issue type, you can add whatever fields you like, let your users fill it and then use it as input for workflow after sanitisation. That was the first step towards self-service :smiley:

1 Like
8

At this point, it would take a lot for them to open up a port to our AWX server. However, it’s my hope that we can get there someday.

I would love to be able to just have Gitlab send a webhook to activate a job.