Prior to 1.5.4 we kept default variables in the playbooks group_vars/all.yml and overrode in the group specific group_vars, followed by the inventory
We have worked round the 1.5.4 → pre 2.0 precedence but do struggle with default values that don’t aren’t defined in a role. I appreciate that that is the preferred approach but this would mean that we would have to include the role default everywhere we use it.
I have situation whereby I want to override a variable stored in group_vars/some_group.yml in an inventory. We have informix_server_host which on most of our ec2 environments defaults to localhost. On a handle for boxes I need to add the db server host. I would like to set a localhost default and override in the specific inventory but this isn’t possible.
If I remove the value from group_vars/ I need to set localhost in every inventory or set localhost in the role default and include that defaults file in every place this is used.
I would have thought that inventory vars would be one of the most specific behind extra vars and task vars. It feels like we are doing something wrong, but we are forced into duplicating variables everywhere at the moment.
Sorry for waffling but i’d be keen to hear if anybody has a good approach to handling defaults that are common to many inventories but need to be overriden on an inventory basis.
Thanks for the reply. Out of interest do you know the rationale for having inventory related vars having less precedence than playbook. I think this is where our problems lie. We have quite a few different types of deployments for different envs. Different box layout configurations etc. Single box / flat tier on local VMs through to 3 tier many box EC2 environments.
I do use overarching groups (groups of groups) to tailor these different environments but occasionally need to tweak certain hosts. This I would like to happen in the inventory as it isolates the change without needing to add lots of playbook level group/hostvars. Though I can’t override at the inventory level with the current precedence. To achieve this I need to knock the var out of the playbook level group and define in each inventory, which leads to duplication.
I guess its just a case of re-thinking the grouping and specifying more playbook level groups.
Sorry for waffling on. I want to try and follow common best practices for laying things out. I just want to try and reduce duplication as much as possible as it causes all manner of problems.
I'm not sure what the problem is as most of the precedence works the
same as it did in 1.5, it was just 'unspecified' and there have been
subtle small changes over the 1.x versions. In 2.0 we decided to make
it as clear as possible what the precedence rules are and avoid
accidental subtle changes in the future.
I was just trying to work out in what situation would you want to have the inventory level group/host vars at a lower precedence than the playbook level group/host vars. It doesn’t really fit with the way we are doing things.
Its something I can work around though though if we are doing something non-standard i’d like to change our approach.
I appreciate that this hasn’t changed, though we did have to change things around the 1.6 point to do subtle changes like you mention.
Plays tend to be more specific than inventory, that is why group/host
vars for plays override inventory ones, but again these are optional.
Ansible is flexible and allows for several workflows, in most cases
though, play group/host vars are in addition to the inventory ones.
I was just trying to work out in what situation would you want to have
the inventory level group/host vars at a lower precedence than the
playbook level group/host vars. It doesn't really fit with the way we
are doing things.
My guess is that the expectation is that you'd typically have one
inventory, which is very consistent, but lots of playbooks, some of which
might want to do things differently. Having a playbook that's generally
consistent, but can be overridden by inventory, seems to me like a less
common use case, but of course if it's the one you've got then it's the
one you care about. :^)
You mentioned that you
struggle with default values that don't aren't defined in a role. I
appreciate that that is the preferred approach but this would mean that
we would have to include the role default everywhere we use it.
I'm not sure what you mean by "have to include the role default". If you
use the role, you get the role default, so could you maybe get what you
want here by just moving the contents of the playbook into a role?
-Josh (jbs@care.com)
This email is intended for the person(s) to whom it is addressed and may contain information that is PRIVILEGED or CONFIDENTIAL. Any unauthorized use, distribution, copying, or disclosure by any person other than the addressee(s) is strictly prohibited. If you have received this email in error, please notify the sender immediately by return email and delete the message and any attachments from your system.
Thanks for the response. We have lots of playbooks & lots of inventories. I plan on creating more groups to better define the behaviour instead of setting at the inventory level.
Re the role issue. I think the problem was around 1.6 group_vars/all vs other group_vars/ precedence changed. We used to have all defaults in group_vars/all.yml and override in the other groups. To try and fix this and move away from having a dumping ground of variables we moved them into relevant roles. The variables may be needed in other playbooks or roles. I.e applications conf.d directory. So to get the variable we would include_vars. We wouldn’t want to run the role just get the variable.
Sorry if I haven’t been clear. Its quite difficult to convey how we do things. We automate everything with ansible. Our system has 50-100 of our own applications built on a stack of similar number of open source tools and apps, data stores etc. So there is a fair amount of coupling.
On the whole Ansible works very well for us. Its just edge cases that get fiddly with the precedence.
Re the role issue. I think the problem was around 1.6 group_vars/all
vs other group_vars/ precedence changed. We used to have all defaults
in group_vars/all.yml and override in the other groups. To try and fix
this and move away from having a dumping ground of variables we moved
them into relevant roles. The variables may be needed in other
playbooks or roles. I.e applications conf.d directory. So to get the
variable we would include_vars. We wouldn't want to run the role just
get the variable.
Ah, yeah; that (setting variables in a role that you want to use in other
roles or plays) may be pushing the limits of how roles are intended to be
used. Hmm.
If you're just interested in filesystem cleanliness (i.e. if "move away
from having a dumping ground" means "don't want an all.yml file with
hundreds or thousands of variables tossed into it"), there might be some
ways to do that. I don't see an obvious way to have a vars file include
other files, alas.
You could try ansible-quartermaster
(https://github.com/caredotcom/ansible-quartermaster) and see if that
helps; it allows you to have arbitrary thing.yml files in its aq.d
directory, which all get merged together to form your eventual collection
of inventory variables. (There isn't really an example of this there, I
could put one togegther if it'd be useful.)
-Josh (jbs@care.com)
This email is intended for the person(s) to whom it is addressed and may contain information that is PRIVILEGED or CONFIDENTIAL. Any unauthorized use, distribution, copying, or disclosure by any person other than the addressee(s) is strictly prohibited. If you have received this email in error, please notify the sender immediately by return email and delete the message and any attachments from your system.
I don’t see how that can be true. The Ansible documentation clearly instructs your to have multiple inventory files as a best-practice. That is, a file per devel/staging/proc/etc. That would mean, each inventory (environment) would have differing variables (IPs, number of app servers, memory settings, etc.). However, you would still use the same playbooks (web server, db server) across all environments.
So, in my mind, at least, inventory vars should have higher precedence than playbook vars, since an inventory/environment will have settings that change even if the same playbook is used.
I’ve got a similar issue that would be solved by inventory variables having a higher precedence.
We have several environments, e.g. test, dev, preprod etc. They use the same roles and groups - web, app, db etc.
But there are some things per environment that vary. For example, as we’re deploying to AWS we might want to create an environment across multiple availability zones or just in a single zone. Or we might want it to be on a single host.
e.g. the default ‘zones’ is ‘a’, but our prod would want ‘zones’ to be a,b,c.
We define each environment via AWS tags, and so we have a separate inventory directory for each environment, each searching for a specific tag.
If we put our per-environment vars in ~/inventory/xxx/group_vars/all/all.yml then they’ll be overridden by vars in ~/group_vars/all/all.yml. So all environments would pick up just ‘a’ for ‘zones’.
We could of course have a ~/group_vars/xxx/ directory for each environment based on tags returned by AWS (e.g. ~/group_vars/tag_prod). But that’s only picked up and applied to hosts within that group, which means that the scripts that actually provision the environment don’t pick up the per-env variables (as there aren’t any hosts until they’re provisioned).
So in cases like this it would be very useful to be able to give inventory variables a higher precedence than group variables.