Playbook interleaving

Archives Ansible Project
1

At some point or another I intend to have a way of interleaving
playbooks to make tasks like setting up both sides of a connection
easier. An example scenario would be:

HostA is a monitoring server. HostB, HostC, and HostD is a newly
provisioned remote clients. In order to bootstrap monitoring of the
remotes there is a playbook pbA that sets up the monitoring client on
the remote hosts, and a playbook pbB that uses some information from
pbA to add the remotes to the list of monitored hosts. The workflow
for multiple clients would be pbA,pbB,pbA,pbB,...

I'm thinking of two ways to approach this - One would be to create a
meta-ansible suite, which would be able to execute multiple playbooks
in any order and pass in arbirtary variables. Another would be to
create some module in ansible that would in effect do a local system
call to run another playbook. The first option would be easier to
implement since it can be done in any language. The second option
would be more integrated with ansible, would help avoid having to
unroll loops (since they can be called like any other ansible module)
and may also be a way to address some of the current feature requests
such as iterating over lists.

I'm looking for input from the community as to whether there would be
sufficient demand/interest for the second option. I think in the long
run I'd like to go with it regardless, but if there is little interest
in such functionality right now I'd just use other methods to script
multiple playbooks in the short term in order to have the
functionality myself faster. Anyway, comments would be appreciated

Regards....

2

I’m looking for input from the community as to whether there would be
sufficient demand/interest for the second option. I think in the long
run I’d like to go with it regardless, but if there is little interest
in such functionality right now I’d just use other methods to script
multiple playbooks in the short term in order to have the
functionality myself faster. Anyway, comments would be appreciated

I’m having difficulty understanding you without better examples and use cases.

I’m not sure what you’re aware of in Ansible land, so to review (and give context to others), the basics are this:

Plays are lists of tasks that run on hosts, and includes are there for reuse. Plays have variable contexts and various parameters and group the orders to the hosts.

Multiple plays can be included per file, if you so chose.

If you have different tasks that occur on different groups of hosts, you can include the same list of tasks, like “tasks/common.yml” on multiple hosts.

It seems that any notion of playbooks triggering playbooks is overkill and you could just run multiple plays in sequence against the different groups of hosts.

Playbooks are supposed to be very deterministic and obvious about what they execute (in other words, auditable), so deciding to run a whole set of playbooks conditionally is right out.

But things were already designed for multinode deployment since day one – for instance, the use case of updating database servers before app servers and then going back to doing something on content servers is pretty well handled, and can all be kept in one playbook file called something like “release_process.yml”, etc.

Ultimately it goes down to understanding the use case to see what you’re really trying to do – and it may be what you want is already achievable, and we should be talking about the “what” not the “how”.

3

I should also add – there’s an outstanding patch for 0.6 that allows any template to query what hosts are in what group, so if a monitoring server had to list out the monitored hosts, it’s pretty
easy to do.

4

The specific cases that I've encountered so far that would be aided by
interleaved playbooks are those where a variable changes based on each
host in an earlier tasks, but later tasks need to be repeated for each
iteration of this variable, or where a task for one host would be made
easier if it had access to the setup variables of another host. For
example, let's say I use ansible to deploy a database to the remote
hosts, and I then want to set up virtualization server with DB slaves
which mirror all these remote hosts's DBs. The steps involved are:

1) Enable replication on the remotes
2) Create a replication user u1 on the remotes
3) Create a DB slave VM
4) Set up an SSH tunnel from each of the remotes to their paired DB slaves.
5) Set up each slave to begin replication

In order to do step 3 in the same playbook as steps 1 and 2 I need some
way of matching the slaves' hostname to those of the masters - e.g.
masterA gets slaveA, masterB gets slaveB, etc. I can do this
dynamically with a script as long as I get the 'masterX' hostnames.
However, since this is run on the virtualization server I cannot use any
of the setup variables for the remotes to fetch the necessary
information when executing the tasks for setting up the slaves (note
that this is not just limited to hostnames, other setup variables like
would also come handy like e.g. eth0.ipv4.address). The 0.6 patch that
you mentioned would make some of this more accessible under certain
conditions, but may fall short in certain situations.

An example pseudo-playbook of what I wish to achieve would be something
like this:

pbA.yml:

5

In order to do step 3 in the same playbook as steps 1 and 2 I need some
way of matching the slaves’ hostname to those of the masters - e.g.
masterA gets slaveA, masterB gets slaveB, etc. I can do this
dynamically with a script as long as I get the ‘masterX’ hostnames.

So you could do that very simply by assigning a hash datastructure right now using something like
vars_files if you don’t want the full membership of the group.

my_slaves = {
“asdf” : [ “one”, “two”, “three” ]
“jkl” : [ “four”, “five”, “six” ]
}

Now, I understand and don’t contest the reasons for not wanting to break
indempotency and auditability in Ansible and thus preferring not to have
playbook recursion, which is why I asked for feedback first and also
suggested the possibility of an independent meta-ansible framework,

No clue what this means, still, so yeah, do not want.