multi stage and variables

Archives Ansible Project
1

Hi!

I tried to read up on everything I could find w.r.t. multi stage environments, but I haven’t found a solution that fits my needs yet.

Let’s say, I have 4 “environments”: dev, test, stage, production
And 2 “kinds” of servers: web, db

To simplify things, I only have on variable, the size of the “ram”.
In “dev”, “test” and “stage” the ram for “web” should be 1GB, and 4GB for “prod” (artificial example to show the issue!)
And in “dev”, “test” and “stage” the ram for “db” should be 2GB, and 8GB for “prod”:

dev test stage prod

±----------+ ±----------+ ±----------+ ±----------+
web | (1GB) | | (1GB) | | (1GB) | | (4GB) |
±----------+ ±----------+ ±----------+ ±----------+

±----------+ ±----------+ ±----------+ ±----------+
db | (2GB) | | (2GB) | | (2GB) | | (8GB) |
±----------+ ±----------+ ±----------+ ±----------+

The roles are centrally defined (say ansible-galaxy), and I cannot use the defaults from the roles.

Possible solutions:

  1. I have an inventory for each stage, but then I have to maintain the “ram” variable duplicated everywhere,
    although I have a useful default value for each “kind” of server. I do not know where to put the default “ram” for “web”
    servers or for “db” servers:

├── dev/
│ ├── group_vars/
│ │ ├── db
│ │ └── web
│ └── inventory

…

└── prod/
├── group_vars/
│ ├── db
│ └── web
└── inventory

  1. I could merge the inventories, like this:

[db-prod]
[web-prod]
[db-dev]
[web-prod]
…

[db:children]
db-prod
db-dev
…

[web:children]
…

[prod:children]
db-prod
web-prod

[dev:children]
…

But the problem is that group precedence is alphabetically, I cannot express that “prod” has precedence over “web”

  1. I could use the fact that group-vars in the inventory vars have lower precedence than group_vars in the file system, so that I define group vars for server “kinds” (db and web) in the inventory file and group vars for environment in the file system. But that feels rather artificial

  2. I could use the alphabetical precedence of group names by using by using names like:

10env_dev 10env_test 10env_stage 10env_prod
20kind_db 20kind_web

but this is quite ugly.

2

If you can't really change the tasks and must use a single variable
named 'ram', I would organize it this way:

# inventory file
[prod-db]
hostA

[prod-web]
hostB

[dev-db]
hostC

# group_vars/prod-db.yml
ram: 8

# group_vars/prod-web.yml
ram: 4

# group_vars/dev-db.yml
ram: 2

Giovanni

3

Hi Giovanni!

Thanks for your reply.

Even if I could rename the “ram” variable (the example is totally artificial):

I would need to add groups like:

[web:children]
prod-web
dev-web

[db:children]

prod-db

dev-db

[prod:children]
prod-web

prod-db

[dev:children]
dev-web
dev-db

And group(-vars) preference is alphabetically! I.e. “db” has a higher precedence as “prod”. Ross Tuck (see [1] above)
has discussed the same solution.

This still leads to quite some repetition, because I need to set the same value in dev & test & stage (and I might even have multiple dev & test environments). And whenever I need to add a variable, I have to remember to add to all these files.

4

So currently all my playbooks will use this (hack):

  • include_vars: “include_{{ item }}.yml”
    with_items: group_names | intersect([‘dev’, ‘prod’, …])

So for “groups” that reference environments, I do not have group_vars, but explicitly included files.
For everything else I can use group vars. I still do not like this.

Especially since the documentation does not explicitly defines this precedence:

5

I am in the process of converting a large system from Chef to Ansible, and this is pretty much my only blocker. I’ve read dozens of posts/articles where this problem is discussed.

So far the answer from Ansible (DaHaan really) has been “I like it this way so tough it up” or “its too late to change it”. But having general variables override specific variables simply does not make any sense. I have seen no argument from the Ansible team that supports this behavior on its merits.

Ansible’s variable setup is so much cleaner than Chef’s, but this really throws a wrench in the works.

6

But having general variables override specific variables simply does
not make any sense. I have seen no argument from the Ansible team that
supports this behavior on its merits.

Is the issue that general variables override specific variables, or just
that there isn't a way to express in the Ansibile INI inventory format
whether "prod" or "web" is more general or more specific?

One thing you could do is write a dynamic inventory script that
understands your particular precedence requirements.

                                      -Josh (jbs@care.com)

This email is intended for the person(s) to whom it is addressed and may contain information that is PRIVILEGED or CONFIDENTIAL. Any unauthorized use, distribution, copying, or disclosure by any person other than the addressee(s) is strictly prohibited. If you have received this email in error, please notify the sender immediately by return email and delete the message and any attachments from your system.

7

Were you able to reach a proper solution to this problem ? I have the exact same problem that I can’t overcome.

8

I am still using the “hack” described above (https://groups.google.com/d/msg/ansible-project/f4jaB5VZ5Dk/0B-ok7XHVMYJ)

I am waiting for Ansible 2.0 and have created an issue (https://github.com/ansible/ansible/issues/12156)

See also the discussion here: https://groups.google.com/forum/#!topic/ansible-project/ari8hggAaZk

9

I see … I’m currently trying to implement something like:

[10staging-webservers]
webserver1-staging ansible_ssh_host=ip

[10staging-dbservers]
dbserver1-staging ansible_ssh_host=ip

[15staging:children]
10staging-webservers
10staging-dbservers

[10production-webservers]
webserver1-production ansible_ssh_host=ip

[10production-dbservers]
dbserver1-production ansible_ssh_host=ip

[15production:children]
10production-webservers
10production-dbservers

[20webservers:children]
10production-webservers
10staging-webservers

[20dbservers:children]
10production-dbservers
10staging-dbservers

Not really sure if this will work or not yet.

10

Maybe not the best way. But for some cases I “misuse” host entries to address this.

So the host entry “web_prod” in the group “web_conf” could have the variable ram with “4GB” and in the group_vars for web_conf ram would have “1GB”. And to access this: {{ hostvars[tier + ‘_’ + stage ].ram }}.

In my case I do this with a generated inventory to manage app and database users/passwords from KeePass. And to separate this the entries have underscores and are in the “user” group.

esco