Gitlab Runner Client side - New module or able to reuse existing modules

Get Help
, , , ,
1

Hi,

I’m co-maintainer of a gitlab-runner role, which is managing gitlab-runners on the client side. For server side, there is an existing community module.

The only drawback of the gitlab role is, that it has to manage a complex toml file: /etc/gitlab-runner/config.toml. The content of this toml looks like this:

concurrent = 1
check_interval = 0
connection_max_age = "15m0s"
shutdown_timeout = 0

[session_server]
  session_timeout = 1800

[[runners]]
  name = ""
  output_limit = 4096
  url = ""
  id = 1
  token = ""
  token_obtained_at = 2025-02-25T22:39:22Z
  token_expires_at = 0001-01-01T00:00:00Z
  executor = "shell"
  [runners.cache]
    MaxUploadedArchiveSize = 0
    [runners.cache.s3]
    [runners.cache.gcs]
    [runners.cache.azure]

[[runners]]
  name = ""
  output_limit = 4096
  url = ""
  id = 2
  token = ""
  token_obtained_at = 2025-03-12T09:28:50Z
  token_expires_at = 0001-01-01T00:00:00Z
  executor = "docker"
  [runners.cache]
    MaxUploadedArchiveSize = 0
    [runners.cache.s3]
    [runners.cache.gcs]
    [runners.cache.azure]
  [runners.docker]
    tls_verify = false
    image = "ubuntu:24.04"
    privileged = false
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    volumes = []
    shm_size = 0
    network_mtu = 0

The file is quite complex and the original author of the role did a really awesome job splitting this file up in chunks, modify it then and put it back together.

Unfortunetely, the file itself is not 100% managed by ansible, but when the gitlab-registation is done, some parts of the file are modified by gitlab itslef (e.g. ids, tokens, …). That means that we cannot simply use a template for creating this file, but need to read in the existing file and only touch the parts which are allowed to be touched on client side.

Because of splitting up the file in ansible, the runs are very slow and are not done in parallel but host by host (ansible related topic).

Now the question for me would be, if it is worth to create an own module to manage this config.toml.

What do you think?

2

We have our own setup for managing config.toml files, but one of the things we do is to use the API to register runners. We then have a config block like gitlab_runner_token[runner_name] that holds the token for that runner and just configure it directly into the configuration file. There’s no need to actually run gitlab-runner register on the runner host machine.

We use this tool I wrote to do the registration (gitlab-runner-ctl register --runner-type … --description "runner_name" will output the registration token)

3

Given that there are not yet any Ansible TOML filters / modules (as far as I’m aware) I guess a external tool could be used to convert the TOML to JSON / YAML and back in order that the role can use YAML configuration? For example:

4

Interesting idea, @chris , but it seems orthogonal to the problem, which is multiple sources of content (Ansible + something else) for one file. Two dogs wagging the same tail, so to speak.

Maybe in combination with @ben.boeckel’s tool (two posts up)?

We have a similar problem with Splunk .conf files. On startup, Splunk conveniently updates plain-text passwords in its .conf files with encrypted versions of the same, with host-specific encryption keys. And Splunk has a forest of directory trees each containing lots of .conf files. Managing those with Ansible is not straightforward.

5

I guess I was trying to suggest that when looking at managing config files like this using Ansible having the ability to read, write and diff them is a starting place… :woman_shrugging:

1 Like
6

@ben.boeckel Does this mean, you completely build up the config.toml in ansible?

7

Yes, it’s just a Jinja2 template.