Following up to my own post, I would like to talk about "reclass",
which is an "external node classifier" that I wrote many years ago
for Puppet,¹ have since ported to Salt,² and am now also considering
for Ansible, because I think it would address some of the issues
I am facing with Ansible.
The purpose of this message is to find those who think similarly to
me, and who may have already figured out Ansible-native ways to do
what I want/need to do. Quite frankly, I'd love to use pure Ansible,
rather than porting "reclass" over.
Here are a number of paradigms/aspects of Ansible. If any of these
ring a bell in relation to problems you are having with Ansible,
then I think you should read on and learn about "reclass":
- The inventory is a list of groups of hosts, and hosts may appear
in more than one group. The inventory is not a group of hosts
with associated groups.
- Groups can be nested, but a group can only ever have one parent
(no multiple inheritance);
- Variable precedence depends on group inheritance: host variables
override those variables set in subgroups, which override those
variables set in parent groups;
- If a host belongs to two groups that are not in the same
inheritance graph, and both groups define different values for
the same variable, it is undefined which value the variable will
have for the given host;
- A playbook hard-codes the group of hosts to which it will be
applied. While the set may be reduced using --limit, it is not
possible to apply a playbook to a host not in the target set
(without changing the set);
Let's illustrate the way reclass would work (if I ported it to
Ansible), using the NTP client example.
Let's assume:
1. we have Debian and Fedora hosts, some of which run as VServers
(which don't allow setting date/time, so NTP makes no sense).
The distribution defaults for NTP servers are
(debian|fedora).pool.ntp.org and we like that;
2. our nodes are either in the office basement, or in datacentres
in Munich or Zurich. At the office, we are fine with the
distribution defaults, except the host "red" should use
"example.org". In the datacentres, we would like to use
(de|ch).pool.ntp.org instead. Yes, this is maybe a bit
contrived, but it'll do as an example.
Reclass works based on the concepts of nodes and roles, but since
"role" is already taken within Ansible, let's use "class" instead.
Conceptually, classes are similar to Ansible host groups, but
I think it's better to keep them separate for now.
Fundamentally, nodes and classes use identical data structures with
three fields — classes, playbooks, and variables. Their difference
is only in the way they are processed by "reclass".
A node usually only defines a set of classes, and optionally
variables (host_vars), but may also define playbooks directly, e.g.
for quick tests or single-host exceptions. A node is identified by
its FQDN (inventory_hostname).
A class defines other classes from which it inherits, playbooks that
apply to this class, and variables (group_vars). A class may also
just define variables to override other classes, as we shall see
shortly.
Therefore, reclass deals with tree graphs rooted at a node (a host),
while all other vertices in that graph are classes. The further away
a class vertex is from the root node, the more generic the class is
said to be, e.g.
node
/ | \ `--- backuppc.client
debian@wheezy | `--- postfix.satellite
/ |
debian munich
/ \
unix molly-guard
/
ntpclient
For each node, reclass performs a depth-first walk of all the
classes (in order of their definition), and their parent classes.
When it reaches a leaf (a class), it reads the list of playbooks and
stores the variables defined. As it ascends back up the tree, it
merges the set of playbooks and the set of variables.
Put differently: the last class to set a variable takes precedence
over earlier classes, and the node has the final say. In the above
graph, if all classes defined $foo, then the final value of $foo
would be whatever backuppc.client set, unless overridden by the node
itself. If both the 'debian' and the 'munich' class defined an NTP
server, the server from the 'munich' class would win.
Let's introduce six hosts for our example.
black Debian Munich
yellow Debian Munich VServer
blue Debian Zurich
white Fedora Zurich
red Debian Office ntp_server: example.org
green Fedora Office
Here's what the reclass data structures would look like to achieve
our NTP client goal (not as complex as the graph above). First, the
classes:
--- ---
name: unixnode name: vservers
playbooks: playbooks:
- ntp_client - ~ntp_client
--- ---
name: debiannode name: fedoranode
classes: classes:
- unixnode - unixnode
playbooks: playbooks:
- apt - yum
variables: variables
- ntp_server: debian.pool.ntp.org - ntp_server: fedora.pool.ntp.org
--- ---
name: hosted@munich name: hosted@zurich
playbooks: playbooks:
- motd@servus - motd@gruezi # whatever …
variables: variables:
- ntp_server: de.pool.ntp.org - ntp_server: ch.pool.ntp.org
And now the nodes:
--- ---
name: black name: yellow
classes: classes:
- debiannode - debiannode
- hosted@munich - hosted@munich
- vservers
--- ---
name: blue name: white
classes: classes:
- debiannode - fedoranode
- hosted@zurich - hosted@zurich
--- ---
name: red name: green
classes: classes:
- debiannode - fedoranode
variables:
- ntp_server: example.org
Since the 'hosted@' classes appear later in the classes list for
each node, their variables take precedence, and so the NTP servers
are set according to location, unless there are no 'hosted@'
classes, when the values from the distro-classes are used. In the
case of "red", the host-specific variable definition trumps
everything else.
And since the 'vservers' class negates the ntp_client playbook, it
would cause that playbook to be removed from the list of playbooks
for the host "yellow", since the class is listed after 'unixnodes'
* * *
At this point, I would be really curious to hear from people who
think alike, and how they do it.
Or if you think that I am completely down the wrong track, tell me
why.
As I said before, I have better things to do that to port reclass,
but I will port it to Ansible if I cannot find another way to
achieve the paradigm I am striving for.
If I were to port this to Ansible, then it would be an external
inventory script. It would put hosts into groups that correspond to
playbooks, e.g.
"ntp_servers_hosts" : ['blue','red','green','white','black']
"apt_hosts" : ['blue','red','green','black','yellow']
"yum_hosts" : ['green','white']
Finally, I would create a playbook for each of these groups,
targetting that group, and also a site playbook that combines them
all.
Thoughts?
-martin
Footnotes:
(¹) http://projects.puppetlabs.com/projects/hiera/ seems like it
grew out of the ideas behind reclass, but I had left Puppet
before Hiera was started, so I don't know for sure.
(²) https://github.com/madduck/salt-reclass/blob/master/README
