Allow a more advanced serialized deploy for multiple clusters at once

Archives Ansible Developer
1

(I’m reposting this comment, I noted within another thread (on -devel), but where it was slightly off topic and overlooked.)

I would like to be able to more easily target multiple sets of clusters within the same playbook in a serialized way, e.g. applying the same role to deploy different apps.

Given say 100 tomcat clusters (wit each containing two or more nodes), deployed with 1 set of roles, in one set of playbooks, targetting the tomcat group.
All are deployed in the same way, but with different apps, parametrized via the inventory, I need to deploy one node at a time.

Now, this is possible with serial: 1 of course, but when a deploy consists of upgrading several related application, it would be a big benefit to be able to deploy several clusters at the same time, but each node within a given cluster 1 at a time.

So, functionally, that would be something like “serial: per node in cluster”.

Explained code wise, this would translate into runner using serial batches, where the first batch contains ‘all the first nodes in the cluster’, the next batch all second nodes, and so on.

I’m not sure ho wthis could be easily defined though. Given all nodes of a paticular cluster are part of a particular application group, and those application groups would be part of, say, a tomcat group, parent/grandparent group membership could be looked at, but so far I couldn’t think of a decent way to model that.

Thoughts?

2

Sounds like a ‘cluster’ concept/directive would need to be added, kind of a ‘serial group by’, I can certainly see the use for it but I’m not sure how that will affect runner and many other dependent facilities like play_hosts, serial fail threshold, etc.

Currently a pretty cumbersome workaround would be running the same play in parallel anasible invocations per ‘cluster’.

3

I've been doing parallel ansible invocations per cluster, but I don't
have that many clusters. I suspect you'd want something that's
explicitly a meta-playbook for this, for expressing synchronization
points.

4

One of the reasons we haven’t addressed this yet is the output would be potentially very messy for a CLI tool.

It’s something that is already achievable with multiple invocations of job templates in Tower, for instance, and has a good structured way to look at the results of jobs effecting previous hosts.

We may also do job dependencies in the near future for instant multi-triggering.

But yeah, why this is not a thing is mainly a question of output formatting with all the background processes going on.

5

One of the reasons we haven't addressed this yet is the output would be
potentially very messy for a CLI tool.

​Now you are just replying to Henry's remark on doing parallel
ansible-playbook invocations.
The idea I was proposing is purely about how to group batches of concurrent
runs in runner().

But yeah, why this is not a thing is mainly a question of output formatting

with all the background processes going on.

​So no, there is absolutely NO problem with regards to output formatting
here.​

6

I was replying to you, though I didn’t really understand your first post apparently.

I still do not :slight_smile: