# Ansible Design Pattern Question

**URL:** <https://forum.ansible.com/t/ansible-design-pattern-question/11598>\
**Category:** Ansible Project\
**Created:** [June 4, 2012, 9:31am UTC](https://forum.ansible.com/t/ansible-design-pattern-question/11598 "2012-06-04T09:31:05Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jonathan\_Palley](https://avatars.discourse-cdn.com/v4/letter/j/dc4da7/32.png) [@Jonathan\_Palley](https://forum.ansible.com/u/Jonathan_Palley)\
**Post date:** [June 4, 2012, 9:31am UTC](https://forum.ansible.com/t/ansible-design-pattern-question/11598/1 "2012-06-04T09:31:05Z")

</div>

All -  
&nbsp;&nbsp;I'm trying to get my head around "the ansible way" for organizing  
things. I greatly appreciate the design goals of ansible and find it  
mirrors largely how I think; but I still am a bit confused about  
proper use of variables, multiple data centers and have yet to find a  
definitive "example/template".

So, here's a first attempt at how we've organized thing. Its a bit  
long because my plan is to take this description with accompanying  
playbooks and make it public as an example/template for others like us

But, I want to first get confirmation this is the "right" way to do  
things and help on a few issues :).

Anyhow, here's the complete story and accompanied thinking.

We have a cluster of machines, as I imagine so does everyone else in  
some manner or form.

So, we defined how it looks in the hosts file. The groups are  
different roles/functionalities (say web-app, db-master, cache - we  
have about 15). We use groups of groups for machines that have shared  
functionality. Its beautiful because we can look at one file and  
clearly see, without distractions, exactly what machines we have and  
how they are grouped.

Now, we write playbooks to setup and/or maintain different aspects of  
this infrastructure. For example, there are "all" playbooks: a hosts  
playbook that copies a shared host file to all the machines; an  
iptables playbook updates the iptables config file (using {% if 'blah'  
in group\_names %}) and so on. We also have a general "base\_setup"  
playbook that includes all the tasks from these specific "all"  
playbooks.

Then we have role specific playbooks that achieve things we might want  
to run on a role; things like 'configure webserver', 'deploy  
application x code', and so on.

For each role we have a number of task files (i.e. things like  
install, config, reload). Each role then has playbooks to achieve  
different objectives based on the tasks. The simplest example is a  
basic webserver:  
Tasks: install, config, reload  
Playbooks: setup (includes install, config, reload tasks), reconfigure  
(includes config, reload tasks)

A more complex example would include playbooks to deploy new code from  
git and so on.

When we setup a new machine we assign it to a group(s)/roles, run the  
base\_setup playbook and then run the associated playbooks based on its  
roles.

If we change something (like say the hosts file or webserver config),  
we run that specific playbook. If we want to deploy new code, we run  
the deployment playbooks.

Compared to where we came from (i.e. 'chef') it all feels very sane  
and logical. One file defines what servers we have. One group of  
files (playbooks) define aggregate objectives (plays) we want to  
achieve on these servers. Another group of files (tasks) has the  
specific task.

Yes, we need to make sure we run the right playbooks when we update a  
config, but this is one command and we always feel like we know  
exactly what's happening.

Is this correct so far?

Now, there are a few things we haven't quiet figured out:  
a) How to deal with multiple clusters. The simplest case is having a  
staging and production cluster. We have that plus ones in different  
countries.

Based on how we organized things above, we create different hosts  
files. The problem with this is we need to have different  
configuration files (in some instances) or different variables  
depending on which cluster we are deploying to. How to achieve this  
in a simple clean way?

My intuition is to have host file level variables (i.e. at the top of  
the host file say: cluster=cluster\_name...but I feel like we are  
missing something.

It \*seems\* the way ansible is designed is to have one host file with  
different clusters organized into different groups. But, I can't  
figure out how to get that to work. Let's say we have a webserver  
with production and and staging. We \*could\* group like this:  
[production-webservers]  
server1  
[staging-webservers]  
staging1  
[webservers] productions-webservers, staging-webservers

We could then ask if the server was in the staging or production  
group.

But, what about the "deploy play"? I want to be able to say: "run the  
deploy play on production servers". Instead, I would need to have two  
copies of this play? It seems to get rather messy rather quickly.

b) Knowledge of other servers. Often times in configuration files, we  
need to enumerate over a list of servers. For example, we have a play  
to configure a munin master. The master config points to all of the  
other servers. Right now, for each cluster I have different config  
files and each time a server is added we need to modify the config  
file. This is very repetitive.

I understand the goal to keep ansible very simple and not have complex  
callbacks and so on. However, to the extent that groups are a key  
construct, I keep running into the simple idea that if I could query/  
access the host structure in variables the code would be much  
simpler. Am I thinking in the wrong direction?

Thanks!  
JP

---

<div class="post-metadata">

**Author:** ![Chamberlain\_Darren](https://avatars.discourse-cdn.com/v4/letter/c/ee7513/32.png) [@Chamberlain\_Darren](https://forum.ansible.com/u/Chamberlain_Darren)\
**Post date:** [June 4, 2012, 2:00pm UTC](https://forum.ansible.com/t/ansible-design-pattern-question/11598/2 "2012-06-04T14:00:49Z")

</div>

\* Jonathan Palley \<jonathan at brainpage.com\> [2012/06/04 02:31]:

> It \*seems\* the way ansible is designed is to have one host file  
> with different clusters organized into different groups. But, I  
> can't figure out how to get that to work. Let's say we have a  
> webserver with production and and staging. We \*could\* group like  
> this:
> 
> [production-webservers]  
> server1  
> [staging-webservers]  
> staging1  
> [webservers]  
> productions-webservers, staging-webservers
> 
> We could then ask if the server was in the staging or production  
> group.
> 
> But, what about the "deploy play"? I want to be able to say: "run  
> the deploy play on production servers". Instead, I would need to  
> have two copies of this play? It seems to get rather messy rather  
> quickly.

My group is still experimenting with our ansible setup, but we've  
solved that problem by decoupling hosts and playbooks.

We're not using a canonical hosts file; instead, we have a series of  
host files, each containing all the hosts in a particular cluster.  
To use your example, we would have host files for  
production-webservers and staging-webserervers, and each would  
contain the list of hosts exactly as if they had been sections in a  
single host file. We set up our playbooks to operation on hosts:  
all, and select the right cluster of hosts by passing -i to  
ansible-playbook.

This is a lot to type from the command line, but we're fronting it  
with Jenkins, so we have it configured with a drop-down list of host  
clusters, so the person running the job simply selects the group to  
run on, and the playbook is applied to all hosts in that cluster. To  
simplify the Jenkins front end, we've got all our host files in a  
subversion repository, and Jenkins reads "tags" from  
[http://svn/configs/clusters](http://svn/configs/clusters) at Build time and generates the list of  
available clusters from that. As long as we keep the subversion repo  
up to date -- and we do, because we use it for other things as well  
-- the Jenkins job is up to date.

(We're still experimenting with our setup, but if we continue to go  
in this direction, we'll likely add a command line tool that,  
instead of invoking ansible-playbook directly, invokes the jenkins  
job which invokes ansible-playbook, which would then handle  
authentication, logging, keep our ansible install up-to-date, and  
all the other things that Jenkins does for us.)

---

<div class="post-metadata">

**Author:** ![Fred\_Alger](https://avatars.discourse-cdn.com/v4/letter/f/c89c15/32.png) [@Fred\_Alger](https://forum.ansible.com/u/Fred_Alger)\
**Post date:** [June 5, 2012, 8:54pm UTC](https://forum.ansible.com/t/ansible-design-pattern-question/11598/3 "2012-06-05T20:54:22Z")

</div>

> All -  
> I’m trying to get my head around “the ansible way” for organizing things…
> 
> Is this correct so far?

Sounds sensible to me.

> Now, there are a few things we haven’t quiet figured out:
> 
> a) How to deal with multiple clusters. The simplest case is having a  
> staging and production cluster. We have that plus ones in different  
> countries.
> 
> Based on how we organized things above, we create different hosts  
> files. The problem with this is we need to have different  
> configuration files (in some instances) or different variables  
> depending on which cluster we are deploying to. How to achieve this  
> in a simple clean way?

I’ve handled this in my playbooks by setting ‘hosts’ to the ‘groups’ variable in them:

- hosts: $groups

Then, when I run ansible-playbook I pass --extra-vars=“groups=production”. The same playbooks can be run against staging by passing --extra-vars=“groups=staging”.

> b) Knowledge of other servers. Often times in configuration files, we  
> need to enumerate over a list of servers. For example, we have a play  
> to configure a munin master. The master config points to all of the  
> other servers. Right now, for each cluster I have different config  
> files and each time a server is added we need to modify the config  
> file. This is very repetitive.
> 
> I understand the goal to keep ansible very simple and not have complex  
> callbacks and so on. However, to the extent that groups are a key  
> construct, I keep running into the simple idea that if I could query/  
> access the host structure in variables the code would be much  
> simpler. Am I thinking in the wrong direction?

You may want to look at writing your own external inventory script and use some existing configuration database.[1] I know that ansible exposes variables from other hosts while running a playbook[2], but I’ve not yet had a need.

[1] [http://ansible.github.com/api.html#external-inventory](http://ansible.github.com/api.html#external-inventory)  
[2] [http://ansible.github.com/playbooks2.html#accessing-information-about-other-hosts](http://ansible.github.com/playbooks2.html#accessing-information-about-other-hosts)