Var precedence

Archives Ansible Project
21

also sprach Michael DeHaan <michael@ansibleworks.com> [2013.06.13.1719 +0200]:

Ansible intentionally lacks vars_scoping.

It's all part of the simplicity of things to not have to have
conversations about scoping.

I am really failing to see your "simplicity of things" argument, and
I also doubt that there are really people who *want* *exactly* the
current behaviour, namely:

  - roles (i.e. reusable, sharable code) can overwrite variables in
    playbooks, and even the ones that are very specific to the
    inventory (host_vars, group_vars). This means that using a role
    may well break other, completely unrelated parts of my setup;

  - role variables themselves can all be overwritten by other roles
    and vars_files, not only through parametrised role includes. You
    are advocating a soft, "by-convention" remediation that is
    functionally equivalent to scoping, but optional and
    error-prone;

  - roles basically do not have an interface and do not obey any
    form of imperative paradigms or closure. It feels a bit like
    the re-use you are trying to achieve with roles is equivalent to
    giving root access to the developers of tools I want to use;

  - there is no way to find out what may be tempering/overwriting my
    variables after all variables from all over the place have been
    munged in the global namespace according to precedence rules
    that do not obey the "most-specific-wins" principle. In fact,
    I'd argue that the way variable resolution takes place is
    actually almost non-determinstic in the event of multiple,
    conflicting defintions;

  - role include parametrisation syntax is different from playbook
    include parametrisation syntax. The former, requiring me to
    enter ascii-representations of data structures feels like a hack
    at best;

Who could possibly want all of that?

I recognize that this is different from what some people want, but
at the same time being *exactly* what some people want.

Good software, especially "simple" software, hard-codes no
assumptions; it provides flexible interfaces and leaves all the rest
up to the user.

22

Ansible intentionally lacks vars_scoping.

It's all part of the simplicity of things to not have to have conversations
about scoping.

I think you're having them. :slight_smile:

If you want something named foo_port, don't name the variable port, etc.

Assuming no scoping, then having conventions like "prefix role
variables with the role name" seems like the best.

But even that doesn't help me understand what to expect if I use a
parameterized role more than once.

E.g. I set "myrole_foo" variable with a default of "bar" in the
role/myrole/vars/main.yml file. Then I have playbook that call the
role once overriding "myrole_foo" and once without:

    - { role: myrole, myrole_foo: baz }
    - { role: myrole }

In the second role invocation, is "myrole_foo" the default "bar" or is
it "baz" because it's a global overwritten by the first role
invocation? From what you said about scope, I think it's going to be
"baz".

If so, then the global scope *severely* restrict the ability to have
reusable roles with defaults. Roles have to be fully parameterized to
ensure the globals are set correctly for each invocation. That's not
DRY.

I realize getting scoping right is hard, but I think it's worth it to
figure out the right way to make it work for ansible.

David

23

Just wondering: have you guys considered making variables immutable once set?
This approach is used by the ant build tool (see [1]), and it seems to me that it might make reasoning about playbooks easier and fit quite naturally with the sequential execution model that ansible follows.

Just a thought,

  • Seb

[1] https://ant.apache.org/manual/Tasks/property.html

24

Yeah guys, I understand you don’t like things.

Ultimately, if you feel this way, it might not be for you.

There is precedence as defined in the docs.

25

BTW, if you pass a variable to the role on the role line, that’s not used later.

I guess I shouldn’t say there’s no scope, I should say that vars_files do not apply only to their specific role.

This is not true of parameters passed to roles, those only apply there.

26

BTW, if you pass a variable to the role on the role line, that's not used
later.

That's good to know!

I guess I shouldn't say there's no scope, I should say that vars_files do
not apply only to their specific role.

So now I'm confused. You said Ansible intentionally lacks vars
scoping, but you do appear to have vars scoping for roles.

It would be really helpful to get some more clarity around all the
different ways that variables can be set and the precedence between
them now that roles are in the picture.

What I get from the docs is this:

* inventory variables -- host vars and group vars

* playbook variables -- vars_file, extra vars on command line, vars in playbook

* "set during the play" -- register option, set_fact module, custom
facts returned from modules

If I understand you correctly, a role vars file is equivalent to a
playbook vars_file (but what precedence between them?).

And you said that parameters provided to a role invocation survive the
role invocation only. But do they have more precedence than anything
else? Or only more precedence than other things at the playbook
level? Or something else?

Unrelated to roles, do register variables survive the play? set_fact
is defined as surviving the play and collected facts clearly do.

Part of what I'm struggling to grok is the interaction between
"precedence" and "survives X part of process".

David

27

It’s been a busy day, roles and tasks can be passed variables, inventory scope is also different than things defined in variable files.

So I was wrong to say there was not scoping.

I initially understood this thread to be about something like having arbitrary levels of scope and nesting, which we intentionally do not have.

It is very important for me to persue a modelling/infrastructure-as-data approach as to being a system that needs to discuss “clojures” and other things, which I believe is a major problem in the automation tools space that excludes users.

What we do is it make it easy to describe what you want to do, and we don’t dwell on the terminology quite so much :slight_smile:

registered variables are available for the rest of the playbook run, and are accessible for other hosts via “hostvars”, which is covered in the docs.

28

also sprach Michael DeHaan <michael@ansibleworks.com> [2013.06.13.2235 +0200]:

It's been a busy day, roles and tasks can be passed variables,
inventory scope is also different than things defined in variable
files.

So I was wrong to say there was not scoping.

I initially understood this thread to be about something like
having arbitrary levels of scope and nesting, which we
intentionally do *not* have.

There are two issues at hand:

  1. some of us believe that a role's vars/main.yml file should
     define defaults at a very low level of precedence, i.e. *below*
     the inventory variables;

  2. some of us believe that there ought to be no global scope at
     all (thereby eradicating (1.)), but that each role should have
     its own scope and variables set within a role cannot ever leak
     outside the role. Only through role parametrisation can those
     variables be overridden, although I feel it would be better if
     the role author could specify a subset of variables that are
     changeable;

I initially came at things from the point of view of (1.), asking
for counter-arguments, and the recent comment on the badness of
the global scope and the goodness of thinking about roles as an
imperative paradigm helped me move on to (2.).

As I attempted to illustrate in
msgid:<20130613155308.GB5350@fishbowl.rw.madduck.net>, I think the
lack of scoping of role variables is a serioues problem. I hope that
you are not going to shut the doors on this one and say that "while
some might not like it, others do, and I won't change it", because
that would be a shame.

It is very important for me to persue
a modelling/infrastructure-as-data approach as to being a system
that needs to discuss "clojures" and other things, which I believe
is a major problem in the automation tools space that excludes
users.

Closure is not that hard of a concept and it's always a question of
whom you target. I think it's reasonable to assume that anyone who
seeks to control large numbers of systems with an automating tool
such as Ansible can be expected to know basic imperative paradigms,
including common closure/scoping, and precedence rules.

I'd even go as far as to say that if anyone approached a set of
machines with an automation tool without that knowledge, they would
do really well long-term investing time now to learn these basic
principles.

Excluding users is never good. But for the same reason as I don't
think that tools like webmin or PHPMyAdmin ("web-based root over
PHP") should ever be touched, I don't think that automation tools
should be written with sympathy for dummies.

29

  1. some of us believe that a role's vars/main.yml file should
     define defaults at a very low level of precedence, i.e. *below*
     the inventory variables;

I can easily see how different people might reach different
conclusions on that point, simply on what they see as the most
specific element of their orchestration. Inventories and playbooks
are a many-many mapping, so depending on whether you have more of one
or the other, it's pretty arbitrary which seems more specific to any
person.

That doesn't bother me as long as it's clearly documented.

Personally, I think such variables should be orthogonal in practice.
Inventory vars should be about the inventory. Putting playbook (or
role) defaults there leads to global variable spaghetti. I don't mind
taking an inventory variable and passing it explicitly to a role
parameter because that makes the code very clear.

  2. some of us believe that there ought to be no global scope at
     all (thereby eradicating (1.)), but that each role should have
     its own scope and variables set within a role cannot ever leak
     outside the role. Only through role parametrisation can those
     variables be overridden, although I feel it would be better if
     the role author could specify a subset of variables that are
     changeable;

I don't think there has to be no global scope. On reflection my
concern is mostly about whether re-usable constructs (roles and
includes) that take parameter get dynamic scope for their parameters
or not. (For anyone not clear on "dynamic scope", see
https://en.wikipedia.org/wiki/Dynamic_scope#Lexical_scoping_and_dynamic_scoping.)

The use of global scope for most variables -- particularly collected
facts -- does makes it easy to sidestep explicit parameter passing and
returns. I get why that is "simpler" for most common tasks.

As I said in some other thread, it does mean having to be more careful
about variable names. For instance, I've taken to always using the
role name as a prefix for role variables. That's fine. It's a bit
verbose, but it works.

David

30

There’s really not a need for variable immutability actually.

What happens is the global things, like vars_files, apply globally, and setting inventory variables and facts does not stomp into the global scope, but are stored on a per host basis. See “hostvars” and the like in the docs.

Yeah, I mispoke when I said there was no scope :slight_smile: Consequence of posting during AnsibleFest when I had limited time.

I meant to say we don’t have anything like block scoping on purpose, because that’s confusing :slight_smile:

31

Hi,

  1. some of us believe that a role's vars/main.yml file should
     define defaults at a very low level of precedence, i.e. *below*
     the inventory variables;




Sorry for bringing up the old thread, but what do you all think of introducing "vars_defaults"

in plays that would be similar to "vars_files", but would have either the lowest possible priority

(lower than "group_vars/all") or at least lower than "vars" (above all inventory variables)?




If roles would than (maybe configurable behavior), instead of "vars_files: .../vars/main.yml"

implicitly use "vars_defaults: .../vars/main.yml" this would solve all precedence issues. Or maybe

roles could implicitly use "vars_defaults: .../vars/defaults.yml" and "vars_files: .../vars/main.yml".



Use case for a couple of re-usable roles each with its own "vars/main.yml" that all define the same

variables and you want to override most of them would result in unnecessary large play, because

you need to specify role parameters over and over again. Eg.:



- hosts: xxx

  roles:

    - role: net_basic
32

also sprach GW <gw.2013@tnode.com> [2013.06.20.1054 +0200]:

Sorry for bringing up the old thread, but what do
you all think of introducing "vars_defaults"

I think this is a cool idea:

  - vars_defaults in addition to vars and vars_files in playbooks;
  - vars/defaults.yml, or defaults/main.yml to populate that from
    a role;
  - vars_defaults would have the lowest precedence, i.e. below
    group_vars.

However, I could still imagine newcomers being confused by the link
between vars/main.yml and vars_files. Once that is understood, the
nomenclature makes sense, i.e. vars/*.yml are all vars_files. But
then we don't want to have a special case for defaults.yml…

So maybe defaults, defaults_files, and defaults/main.yml?

33

I worry about introducing an additional level of complexity and another place to put variables.

group_vars/ is a great place to put defaults, and in 1.2 and later you can store those directories alongside your playbooks.

Basically our goal should be to have as few conceptual places to put things as possible to keep it easy to remember.

This is why “roles/” are just some magic on top of things we already had in previous versions of Ansible.

34

vars_fallback, might be a more apropos name (last resort).

35

It’s still another keyword.

Not adding anything new here for now.

–Michael