Remove vars directory from ansible-galaxy generated roles?

Archives Ansible Developer
1

Concerning the role generated by ansible-galaxy:

I question the inclusion of the /vars directory in the generated role. Populated /vars directories in roles found in the wold are a barrier to reuse. While it is great that variables defined in /vars are scoped to the role, I believe the same results can always be achieved by including variables in /defaults when developing roles for others to used.

Populating the /vars directory should be left completely to the end user of a role. A user who downloads a role should be free to add a /vars directory if they choose (or symlink to some vars directory or do anything else they please - these vars are for the end user). A user adding /vars should also not be tracked in version control. Currently, /vars are frequently included with roles that I see around github and tweaking them to customize the behavior of the role is a change that is tracked preventing clean pull upgrades. I believe /vars should be added to the .gitignore of the generated role.

As far as I know, placing role variables in /defaults can always achieve the same results as placing variables in /vars. But adding some discipline to make role developers use /defaults provides benefits for reuse and end users of roles.

To be clear, Ansible should absolutely still read variables from /vars, just encourage that populating /vars be left to role users. Too many role developers are using /vars. If the Ansible community doesn’t agree that this would be beneficial (which is fine, that’s up to you guys), our tooling will probably be considering only a subset of roles (those that exclude /vars from VCS) to be “valid”.

2

I’d generally agree that defaults are a good place to put things, though if you want to be absolutely certain you set a variable, using role defaults means you’re not sure it will take.

I’m not sure I agree that roles should be barred from using vars/, but rather that well named variables in defaults be encouraged instead.

Now here’s the deal.

The ansible-galaxy CLI is good for general scaffolding creation whether you intend to upload or not. It should still create the dir.

The question is probably whether the upload tool might want to omit a suggestion, or rather that this is just covered in documentation.

3

I’m not entirely sure that I understand your post…

Concerning the role generated by ansible-galaxy:

Technically the roles were generated by their authors, you load them from the galaxy repository using ansibe-galaxy.

I question the inclusion of the /vars directory in the generated role. Populated /vars directories in roles found in the wold are a barrier to reuse. While it is great that variables defined in /vars are scoped to the role, I believe the same results can always be achieved by including variables in /defaults when developing roles for others to used.

So, you are saying that people writing roles should only include vars set in /defaults and not in /vars. I would suggest that most people developing roles are developing roles for themselves and are then sharing their roles as a way to give back to the community. As a result they are sharing a sample /vars file.

Populating the /vars directory should be left completely to the end user of a role. A user who downloads a role should be free to add a /vars directory if they choose (or symlink to some vars directory or do anything else they please - these vars are for the end user). A user adding /vars should also not be tracked in version control. Currently, /vars are frequently included with roles that I see around github and tweaking them to customize the behavior of the role is a change that is tracked preventing clean pull upgrades. I believe /vars should be added to the .gitignore of the generated role.

That’s nice… I would suggest that when I create a role for myself, to scratch my own personal itches, I should be free to include /vars as that makes sense for what I want to do. If I then decide that I want to publicly share my work then I should be allowed to include the /vars file if that is what I want to do.

As far as I know, placing role variables in /defaults can always achieve the same results as placing variables in /vars. But adding some discipline to make role developers use /defaults provides benefits for reuse and end users of roles.

To my mind /defaults has different connotations than /vars… You might disagree but please follow along and see if this makes sense…

Let’s say I have developed a role, Widget, and that role needs to configure something to talk to another server. I don’t know what server name is going to work in your environment, but I can pretty much guarantee that it won’t be whatever worked for me. So If I put that server name in /defaults the implication is that that value will work for you out of the box. It’s a default, not a placeholder. In reality that “default” is not a default at all. So I could document somewhere that you will need to set variable “FooServer” to point to your FooServer, but then you have to remember to do that. If I instead put in a /vars file with “FooServer” set then my assumption would be that you would look at that file BEFORE applying the role to ensure that the contents made sense for your environment.

I realise that this is reading something into the names of the directories that is not actually imposed by the technology, but my assumption would always be that /defaults are just that, default values that should work.

To be clear, Ansible should absolutely still read variables from /vars, just encourage that populating /vars be left to role users. Too many role developers are using /vars. If the Ansible community doesn’t agree that this would be beneficial (which is fine, that’s up to you guys), our tooling will probably be considering only a subset of roles (those that exclude /vars from VCS) to be “valid”.

You are free to do what you want, this is a big world and I think that there should be room enough for everyone… (I could be wrong of course)

I understand where you are coming from, but I do not entirely agree with your conclusions.

Of course I have nothing to do with Ansible (except as a user) so none of this means anything to anyone other than me… :slight_smile:

I hope that you can at least understand why I feel that /defaults and /vars are not the same thing.

Adam

4

But if you git clone a role, the moment you tweak a variable under vars (which is usually tracked currently), you’ve introduced a change that needs to be commited or dealt with before a clean pull can be performed.

I suppose I care about this because users of my tooling don’t want to deal with variable changes causing changes in the version history of the role. It is desired that the role and the variables the user chooses to use (to override defaults) be isolated and versioned separately (we symlink /vars from each role out to another location where all variables are stored).

Yes, it is good that defaults are the lowest priority. The design is quite close to having isolated levels of variable configuration inside the repository and outside, it just doesn’t quite go all the way and encourage or enforce role developers to save /vars for end users.

5

Would you rather that your users had to check to see why the /vars is different now or that they break stuff because they’ve just applied a bunch of new defaults that won’t work for them?

Personally, I would rather have to check that changes weren’t a big issue than have things break because I just applied some new defaults that are useless (or worse, actively dangerous).

Adam

6

@adam

I’m not entirely sure that I understand your post…

Concerning the role generated by ansible-galaxy:

Technically the roles were generated by their authors, you load them from the galaxy repository using ansibe-galaxy.

I question the inclusion of the /vars directory in the generated role. Populated /vars directories in roles found in the wold are a barrier to reuse. While it is great that variables defined in /vars are scoped to the role, I believe the same results can always be achieved by including variables in /defaults when developing roles for others to used.

So, you are saying that people writing roles should only include vars set in /defaults and not in /vars. I would suggest that most people developing roles are developing roles for themselves and are then sharing their roles as a way to give back to the community. As a result they are sharing a sample /vars file.

It makes complete sense that people are developing these for themselves, they’re using /vars, as is totally fine since they are the end user and then they share the result to help the community. Cool. I’m pushing for a little more responsibility from role developers though. When I publish a piece of software that I developed on my machine, I strip out everything that made it suited to just my machine and generalize it, usually though overridable settings. I think about Ansible roles similarly.

Populating the /vars directory should be left completely to the end user of a role. A user who downloads a role should be free to add a /vars directory if they choose (or symlink to some vars directory or do anything else they please - these vars are for the end user). A user adding /vars should also not be tracked in version control. Currently, /vars are frequently included with roles that I see around github and tweaking them to customize the behavior of the role is a change that is tracked preventing clean pull upgrades. I believe /vars should be added to the .gitignore of the generated role.

That’s nice… I would suggest that when I create a role for myself, to scratch my own personal itches, I should be free to include /vars as that makes sense for what I want to do. If I then decide that I want to publicly share my work then I should be allowed to include the /vars file if that is what I want to do.

Sure, its your repo. That’s your right. Share anything you like. But at least with my tools, I’d like to only consider role where the author has stripped out /vars and explicitly designed the role for reuse. Building manager tools for roles is painful when people have their own variables in the source.

As far as I know, placing role variables in /defaults can always achieve the same results as placing variables in /vars. But adding some discipline to make role developers use /defaults provides benefits for reuse and end users of roles.

To my mind /defaults has different connotations than /vars… You might disagree but please follow along and see if this makes sense…

Let’s say I have developed a role, Widget, and that role needs to configure something to talk to another server. I don’t know what server name is going to work in your environment, but I can pretty much guarantee that it won’t be whatever worked for me. So If I put that server name in /defaults the implication is that that value will work for you out of the box. It’s a default, not a placeholder. In reality that “default” is not a default at all. So I could document somewhere that you will need to set variable “FooServer” to point to your FooServer, but then you have to remember to do that. If I instead put in a /vars file with “FooServer” set then my assumption would be that you would look at that file BEFORE applying the role to ensure that the contents made sense for your environment.

I see your reasoning, but I tend to think of default as placeholder if needed. An end user of the shared role Here I would want the server name in defaults to be set to server_name: None or localhost or something that probably isn’t going to work out of the box. The defaults get copied over to /vars (but still not tracked in VCS) and the user can tweak those settings as needed to make it actually work. I’m fine with the defaults all being None.

Reading your description does make me think an example valid /vars (such as the vars that worked for the role developer) would be useful, but comments might be sufficient to indicate what kinds of values are expected or what the choices are.

I realise that this is reading something into the names of the directories that is not actually imposed by the technology, but my assumption would always be that /defaults are just that, default values that should work.

To be clear, Ansible should absolutely still read variables from /vars, just encourage that populating /vars be left to role users. Too many role developers are using /vars. If the Ansible community doesn’t agree that this would be beneficial (which is fine, that’s up to you guys), our tooling will probably be considering only a subset of roles (those that exclude /vars from VCS) to be “valid”.

You are free to do what you want, this is a big world and I think that there should be room enough for everyone… (I could be wrong of course)

I understand where you are coming from, but I do not entirely agree with your conclusions.

Of course I have nothing to do with Ansible (except as a user) so none of this means anything to anyone other than me… :slight_smile:

I hope that you can at least understand why I feel that /defaults and /vars are not the same thing.

They definitely are not the same thing and that’s great. The introduction of separate /defaults and /vars is a great opportunity to isolate default configuration of a role from the configuration done by an end user.

At the end of the day, maybe role developers like having their personal vars in source control because it is easy for them though.

7

To me, the defaults are there to show all the variables that are possible and give default values to some for which it is appropriate. The others can all be None and must be set by the end user in /vars because the role can’t reasonably be expected to work without the user setting them.

8

I guess my question is, why would you ever have to tweak a variable
that's in var/ ?

In my experience, the only things I've been putting in vars/ are
things that are fairly static -- maybe it is a list of packages or a
name of service that is slightly different between Linux
distributions. These vars/ files are loaded conditionally based on
operating system, for example:

- name: Add the OS specific variables
  include_vars: "{{ ansible_os_family }}.yml

I think you have some concerns, but I do think if vars/ is used
correctly, as in a place for setting things that don't need to be
overridden (like my example above), then there's no need to remove
them or not use them in Galaxy.

- James

9

Righto.

I think this is a docs thing that we should make clear in the “about” page for galaxy.

Vars are actually “scope” protected so in a role, you are guaranteed that vars for that role “take”.

Thus if role foo says “x: 2” and role y says “x: 3”, you don’t have to worry about x or y conflicting.

(It is true that “x” lives after, and is accessible in other roles)

10

How does this compare with variables in defaults and what Ansible recommends they contain?

11

So I currently .gitignore vars in roles and symlink to somewhere else on the filesystem that contains a vars file. This works really well with our roles which are small and modular and all do this with git ignoring vars, but it is hard to generalize this for the whole ansible community to use.

Passing in variables to roles is not practical in general (we would have to pass all of them each time we use a role as a dependency) and creating a symlink inside the role to some variable files (/vars was nice because Ansible loads those by default and doesn’t notice that it is following a symlink) that is not .gitignored is a change to the tracked files and has to be committed (which would be undesirable, we want isolation).

Any ideas how this strategy could work with general Ansible roles?

12

Defaults are overridden by any/every other setting. What Ansible recommends (at least in the documentation) is quite clear…

"Ok, so if you are writing a redistributable role with reasonable defaults, put those in the ‘roles/x/defaults/main.yml’ file. This means the role will bring along a default value but ANYTHING in Ansible will override it. It’s just a default. That’s why it says “defaults” :slight_smile: "

I guess my question is where do you put placeholders for other people to configure that are NOT reasonable defaults. Perhaps putting them in /vars/EXAMPLE.yml might be a good suggestion, and then not including a /vars/main.yml… That would allow people to provide an EXAMPLE that shouldn’t cause the issue that concerns you and yet isn’t the reasonable default I expect in /defaults…

Of course that would be a documented preference, not an enforced hard and fast rule, but, hey, what do you think of that?

Adam

13

How does this compare with variables in defaults and what Ansible recommends they contain?

Defaults are overridden by any/every other setting. What Ansible recommends (at least in the documentation) is quite clear…

"Ok, so if you are writing a redistributable role with reasonable defaults, put those in the ‘roles/x/defaults/main.yml’ file. This means the role will bring along a default value but ANYTHING in Ansible will override it. It’s just a default. That’s why it says “defaults” :slight_smile: "

Yes, exactly. This is the strategy.

I guess my question is where do you put placeholders for other people to configure that are NOT reasonable defaults. Perhaps putting them in /vars/EXAMPLE.yml might be a good suggestion, and then not including a /vars/main.yml… That would allow people to provide an EXAMPLE that shouldn’t cause the issue that concerns you and yet isn’t the reasonable default I expect in /defaults…

For things that don’t have reasonable defaults, we still include the variable in the defaults and anything in ansible will override it. The top sections of defaults/main.yml starts with variables like these (required). Defaults with values that work and are fine are called Preset. Some may consider it strange that the defaults will not work out of the box, but a role that requires variables like this won’t work out of the box any way you do it really.

Required - your user vars file must specify the following

valid choices: Hostname string

some_important_thing: None

Preset defaults

valid choices: red, green, blue

favorite_color: “red”

We never include any vars/* in our role repository. Like I said, its actually a symlink. When users first download a role, defaults/* is copied to their vars/* location on their file system and the symlink is setup. The user is free to change variables in vars/* to override the defaults.

The user never needs to touch any file in the role they downloaded. Ok, maybe there is a nice readme he/she wants to peek at, but that is it.

Of course that would be a documented preference, not an enforced hard and fast rule, but, hey, what do you think of that?

I do think documentation style is up to taste. Our defaults/main.yml is pretty self documenting as you can see.

14

Unreasonable Defaults might be a good name for a rock band.

15

Any suggestions for this situation?

So I currently .gitignore vars in roles and symlink to somewhere else on the filesystem that contains a vars file. This works really well with our roles which are small and modular and all do this > with git ignoring vars, but it is hard to generalize this for the whole ansible community to use.

Passing in variables to roles is not practical in general (we would have to pass all of them each time we use a role as a dependency) and creating a symlink inside the role to some variable files > (/vars was nice because Ansible loads those by default and doesn’t notice that it is following a symlink) that is not .gitignored is a change to the tracked files and has to be committed (which > > would be undesirable, we want isolation).

Any ideas how this strategy could work with general Ansible roles?

Some way to symlink in something like /vars to the scope while not being tracked in the vcs.

16

I’m afraid not from me.

While thinking about this general topic the one possibility that struck me about /defaults or /vars when used as a placeholder that you know should be changed by the person running the role is to deliberately set the variable to something specific and actually check it in the play. If it is your placeholder default then fail with a warning message.

Again this isn’t really enforceable… (at least I can’t think of a way for ansible to know if a default is reasonable or a placeholder) but it might be worth suggesting in a style manual. Whether you set the placeholder to “None” or “ChangeMe” or “rm -rf /” is not really my concern, as long as 1) you don’t set it to something that someone might actually want to use and 2) you check to make sure that it is no longer set to the placeholder in your role.

I think that that is the overriding result of this discussion in my mind. I can’t see any way of Ansible or galaxy enforcing many of these suggestions without it coming across as heavy handed. I can think of workarounds for consumers (you could clone the github repository, then have a process to copy the role to your own repository without the vars directory, or even merging vars with default if that is what you want…, you haven’t changed the git clone so no problem updating it, yet you are not having to worry about /vars) or suggestions for role developers “If you have put in defaults then make sure they are in /defaults”, “If you use placeholders variables then set them to something nonsensical and check that they have been changed in the role”… and so on.

Unless the owner of a repository wants to dictate specific rules for inclusion in that repository (e,g, Ansible could say "No /vars in roles in Galaxy) then most of these suggestions are simply that… suggestions. Ansible could actually take it one step further and say “we don’t like this feature we implemented and we are removing it, so no /vars in roles ever moving forward” but again it’s their product.

I’m sorry that I don’t have the perfect solution, If Ansible want to enforce some of these rules they can, to some extent, but until that time the best I can suggest is workarounds for you to implement and style suggestions for developers to follow.

Adam