include_vars: Option for preserving existing variables from being overwritten

Hi all,

in our project we are delivering a set of playbooks and roles to our customer on a regular basis. Our goal here is that the customer should not change our playbooks and roles and that he is able to configure & overwrite everything within his inventory via inventory variables, so that he doesn’t always have to merge his configuration when we deliver a new release to him. Unfortunately, this approach is currently not possible when using the include_vars module anywhere in our code because it overwrites the inventory variables defined by the customer (which should take precedence of course).

I think this is a quite basic use case and I’m a little bit unhappy that there is still no solution for this yet. E.g. it was already discussed within Proposal #21. In that discussion, I proposed a very easy, minimally invasive solution for this: Just adding an option to include_vars which makes it possible to define that already existing variables should not be overwritten by include_vars. Probably, this would also be a helpful option for other use cases as the one described above. For implementation of this approach, I created PR #23738 already longer ago and it’s idling quite a while now. A few days ago, I reimplemented this PR against the current devel branch and added documentation as well as an integration test.

I would really appreciate if some of you may interested in this topic could review this PR (it’s just a few lines of code!) and take part of the discussion how to solve this use case.

Thanks in advance,
Patrick

You would still have issues with set_fact, vars:, vars_prompt,
register, vars_files and inline vars to include and roles, as
inventory is not very high in the precedence.

One way to avoid this is to have client provide extra vars, which has
highest precedence, instead of inventory.

Another way is you provide your vars in a role as defaults/main.yml,
which has the lowest precedence, instead of include_vars.

Thanks for your input, even though I don’t fully agree.

register vars / set_facts: If some “default” configuration options are e.g. gathered by shell commands and registered as variables with set_fact, this can by easily solved by just adding when: var is not defined to the particular task(s). For include_vars this is not an option because you have usually much more vars here and would have to put every variable into the when statement as well.

vars_prompt: I don’t think that it’s a use case to overwrite those variables within the inventory, as the options provided via CLI should always take precedence.

vars_files / inline_vars: As you already wrote, those can be easily prevented by using defaults/main.yml instead.

Meanwhile, include_vars usage cannot be easily prevented as you really need e.g. to write roles with multi-platform support (as already discussed in Proposal #21).

I think extra_vars is not a realistic option for configuring everything, as in big projects people have to overwrite many variables and they also want to store their configuration within files to keep the ansible runs reproduceable.

You can pass the path to a YAML file to -e/–extra-vars, which allows them to keep their configuration within a separate file.

–extra-vars @path/to/file.yml

Thanks, that already helps a lot! Anyway, there are still cases where variables have to be overwritten differently for specific hosts / host groups. So this is still not the overall solution for the issue in general. Regarding the suggestion I made for adding an additional option to include_vars, do you see any disadvantages for adding such a possibility?

Totally agree with @Patrick, who asked this question originally in 2016 here:
https://groups.google.com/d/msg/ansible-project/jFf1KfoWxBA/SKFl8e0gAQAJ

… and still now satisfying solution exists for us role contributors (that care about role portability in larger roles).

That includes the contributor himself, geerlingguy, who had done a proposal …where I added a post summarizing the current status (AFAIU):
https://github.com/ansible/proposals/pull/21#issuecomment-470048538

I can understand if the more satisfying solutions could be tricky to implement in the ansible core, but then it would be helpful to have an estimate of the work required for the various solutions.

Anything else I am missing?