role var_files and variable precedence

Archives Ansible Project
1

The new roles feature in 1.2 provides a very interesting level of modularity and re-usability. While ansible is ‘batteries included’ because of its rich set of modules, roles seem like higher level service definitions (built upon modules).
I have begun concerting over certain higher level functionality (e.g. apache, varnish, etc) to roles and in doing so I leverage the roles//vars/main.yml (vars_file) as the location to define the sane default values for variables. However, I would like to enable users of the role to be able to override these defaults if/as necessary.

I’ve found that vars_files defined at the playbooks level have higher precedence than roles//vars/main.yml, and role params (e.g
roles:

   - { role: foo, param1: 1000 }
have higher precedence than var_files. But what I am wondering is if it makes sense, in the case of roles, to have even inventory variables higher precedent than role vars. My rationale is:

- Roles are discrete re-usable definitions of higher level functionality/capability that could (should?) be shared (I could even see a roles repository at some point down the rode). Providing sensible defaults in roles/<rolename>/vars/main.yml makes the most sense to me.
- GIven the above, inventory variables make sense, and are an easy way to override role vars. Requiring the user of the role to specify the variable override as a role param or even in a vars_file (via the playbook) just does not feel as simple to me. Even if you disagree, having
the option to use inventory vars instead of playbook vars_file and role params seems a good idea.
- Some of us use inventory scripts and our primary means of setting custom vars is via inventory vars. My inventory script accesses a DB which contains not just groups of servers, but also the vars for that specific environment. 

So to summarize, would it make sense to treat role var_files with lowest precedent so they can be easily decoupled from playbooks and yet overridden with inventory variables?
2 
I understand lots of folks have ideas about defaults.  I think, largely, we need to cover this by having some more documentation in the roles section.

Yeah, so if you need to override something in roles/foo/vars/main.yml, passing in the parameterized value to the role works.  This is a great way to do it.  Your role can carry along it's default parameters, but you can ALSO chose to ignore them.  I should have thought about this this weekend but was fighting a small conference-plague, and have since recovered to where my brain is operational again :)


Ansible's policy that inventory variables have lower priority is entrenched, and I'm unwilling to change that because thousands of users rely on it.  It's ok though -- group_vars/all makes such a great place for global site-wide defaults.  By placing defaults in group_vars/all, you can get all of the defaulting behavior you want.  (Set other parameters in a specific group, and a group will override a more general group, and host_vars overrides a group too!).


Why do parameters in playbooks always win?  Since Ansible is designed to be very very auditable, if you see a parameter in a playbook, you know exactly that value is going to be used when you run the playbook, and don't have to hunt through a possibly dynamic (read CMDB/Cloud) inventory source to see where your variable values are coming from.   


I generally think if you find yourself (the royal you, here, us, we?) stuck in a situation where the default policy comes up, try to confine variable definitions to either inventory or playbooks, and you'll find it much easier to think about.   Inventory variables are for where you have things that change from geography, or group of system, etc.   Things that don't change, things like hard coded port values, go in playbooks (roles, etc) and are used directly.

I generally feel that if we made priority tweakable, that becomes more confusing when we grow to read each other's playbooks, and we need to continue to keep it simple.

To me, if there is a variable that gets set from inventory, because it's say something like a regional NTP server, that value isn't a default anymore -- that value is what you want to use.  As such, don't define it in the role.  If you want a site wide default, put that default in group_vars/all, and it will work as such, and group_vars/boston can define a different value to override.
3

Role parameters are a viable option in my opinion. However, with the recent discussion of ‘namespacing’ role vars somewhat the params can look kind ugly, Is there a better way (for apache.port):
roles:

  • { role: ‘web’, ‘apache’: { ‘port’: ‘param_from_role_param_in_playbook’ }}

I do think treating roles as very discrete re-usable components, perhaps download from other sources, is critical to the new role capability. And being able to override their default variables is necessary. Doing that as role params is adequate - as long as the format to do so is fairly clean.

4

Yeah you don’t have to namespace here at all.

The role parameters will clobber what you need.

That discussion was about something like this:

vars_files:

  • one.yml
  • two.yml

where x: 5 in one.yml and x: 6 in two.yml would result in x=6

How does that relate to roles? Because the vars/ directory with roles maps exactly into things being sucked into vars_files.

If you’re parameterizing the role, you don’t have to worry about that, whatever parameter you use will definitely be set.

5

Thanks Michael. You are right, in this case namespace is not required (I over designed :-). I could just quasi namespace by prefixing the string of the rolename to the var to avoid variable collisions (e.g apache_user=www so it does not collide with var named user perhaps applicable elsewhere) and that avoids the gnarly syntax in the role param.

I can certainly make this work. Please do still mull this over though. I see providing defaults in the roles vars_file as a form of self documenting the roles inputs. And making (only) roles vars_file lower precedent does enable people like me using inventory scripts a bit more flexibility (I can store the cfg in my db/inventory source, rather than tweaking the playbook) when I need to override those role defaults.

6

This makes perfect sense. Given this logic, why is it that the vars section
in a playbook has lower precedence than vars_files/roles variables?
Shouldn't a variable defined in a playbook have higher priority than a
variable defined in an external file?

Lorin

7

There’s really no meaning to vars/vars_files having any difference of precedence, the code just needed one or the other.

When I built them over a year ago, I wanted them to be “the same”, and of course that’s technically impossible, so I just picked one or the other. One line of code had to execute before another.

By “variable defined a playbook”, above, I mean’t vars OR vars_files, referring to the “greater playbook”, of sorts :slight_smile:

8

I think it’s a shame that tie went to vars_files instead of vars, because roles variables files work like vars_files, which means they override vars. In particular, this means that if you define vars in a playbook, roles variables will override. This means you can’t define your variables at the top of your play if you want to override role variables.

Here’s one I found really confusing: the pre-task below won’t fire if roles/myrole/vars/main.yml contains activate: false:

9

One option could be a variable at role vars_file scope* that when set inverts the logic in utils.combine_vars(a, b).
Something like in roles/foo/vars/main.yml
default_vars: true

then in utils.combine_vars(a.b) check for default_vars == True and if found reverse it to dict(b.items() + a.items()) #and likewise for merge
so that ultimately the scope where default_vars is set to true has the lowest scope.

*technically this could work at any vars scope - but I think its mostly applicable to role vars scope.

10

Yeah, we aren’t changing scope folks.

Sorry.