Trying to make a playbook extensible by dropping files in

Archives Ansible Project
1

Hi there.

I bothered some people on IRC already, but I understand that this is a niche case and people (understandably…) turned away after providing the same suggestions over and over again.

Overview, goal: I want to have a playbook that is taking input from its role/myrole/files folder dynamically.

I want

  • to add stuff there and ansible to pick it up when I redeploy
  • to remove stuff there and ansible will overwrite the previous results, restoring a consistent state
    All this should happen without changes to the .yml file or a template. Just files/folders.

A specific goal of mine (I … have several places where I need this):

opendkim needs keys (files, think ssl cert keys if you like) and a configuration file to refer to those. My plan is this:

Files below roles/myrole/files:

ls files/opendkim/ -R
files/opendkim/:
keys

files/opendkim/keys:
domain1  anotherdomain  andanother

files/opendkim/keys/domain1:
default.private.enc  default.txt

files/opendkim/keys/anotherdomain:
default.private.enc  default.txt

files/opendkim/keys/andanother:
default.private.enc  default.txt

Now I need a template (of many, I need to transform the list into multiple formats) to generate the follwing file:
(format simplified, not the real/final opendkim format)

domain1 /etc/opendkim/domain1/blaaa
anotherdomain /etc/opendkim/anotherdomain/blaaa
andanother /etc/opendkim/andanother/blaaa

The problem is, I want the dynamic structure (below files/) as a list. That fails over and over again.

Things I tried and discarded:

  • with_fileglob: => Doesn’t work for directories/only lists files and has the problem all with_* loops have (see below)
  • with_lines: ls opendkim/keys => That returns what I want, but has the standard with_* loop issue (see below)
  • local_action: command ls opendkim/keys (or substitute raw for command) => working directory is wrong, doesn’t start/look in roles/myrole/files but in the current shell’s pwd

Having anything but a list with a name (registered, as a variable etc.) cannot be expressed in a template: How am I going to create a template for the same target file with a list as input?

The issue with templates and with_* for me:

template: src=someTemplate dest=mergedConfig
with_items

  • a
  • b

renders the template twice. I want to create a file from the whole list, not each item (and ending up with the last item being rendered).

So - how about registering that list of values I want? So that I can refer to it by name in a following template: action?
Turns out that doesn’t work (well) either. The best I got was this:

I need an action, with_* loops cannot register something on their own. An ‘identity’ module would be awesome

debug or command: echo or … anything useless needs to be put here

debug: msg="{{item}} or whatever}}
register: opendkim_keys
with_lines: ls opendkim/keys

This actually, kinda, sorta works. Ignoring the output that I don’t want, this registers a huge dictionary. My ls output has a couple lines, simple strings. This registers the result of the task which contains a huge amount of overhead (think: What command was run, where, what was the stdout, stderr, resultcode etc. etc.) and information I don’t care about. This is the best I can do so far, but this would mean that I’d need to

  • pass opendkim_keys.results as a list (somewhat okaish…?)
  • access {{item.item}} in that list (try it… that is the value I want, but that is really ugly)

At this point I’m close enough to go for this solution, but that makes the template god awful to look at.

Is there anything I’m missing in general? Can this be done, without hacks? Do you think I shouldn’t do this in the first place? Why?

Thanks in advance,
Ben

2

So this is rather long and by policy I always just answer the first question asked as it’s very difficult for everyone to read an essay, or a response to an essay.

First question is first:

"Overview, goal: I want to have a playbook that is taking input from its role/myrole/files folder dynamically.

I want

  • to add stuff there and ansible to pick it up when I redeploy
  • to remove stuff there and ansible will overwrite the previous results, restoring a consistent state
    All this should happen without changes to the .yml file or a template. Just files/folders.’

If you want to choose a file to import dynamically, you could pass in a variable to “-e” and so something like

vars_files:

  • “get_variables_from/{{ magic }}/foo.yml”

and on the command line do “-e magic=directory123”

I do not understand the question about “remove stuff there and ansible will overwrite the previous result”, so please explain that one further.

Depending on those answers we are proceed to the remaining questions.