Emulating host_vars and group_vars for files

Archives Ansible Project
1

Dear Ansible developers and users,

I really like the groups_vars and host_vars concept of Ansible, and I was thinking of a way to emulate this for files. So here’s an example playbook:

  • hosts: all
    tasks:
  • debug: var=item
    with_first_found:
  • paths:
  • host_files/{{ inventory_hostname }}
    files:
  • myfile
  • paths: “{{ group_names }}”
    files:
  • myfile

This works just fine for a host-specific file, because there can only ever be one directory for a host. it also sort-of works for groups, but there are 2 complications:

  1. the order of the names in group_names does not match the inventory parent-child order; and
  2. I don’t know how to prefix “group_files” to each group name, in order to keep my group files in their own directory.

Can anyone suggest how I can achieve this? Or does this require a new custom plugin?

Anand

2

1. the order of the names in group_names does not match the inventory
parent-child order; and

​There is no way of knowing that, as the inventory does not export that
information to other ansible parts​

2. I don't know how to prefix "group_files" to each group name, in order to

keep my group files in their own directory.

perhaps I miss your point but

  - paths: "group_files/{{ group_names }}"
    files:
            - myfile

?​

3

Hi Serge,

  1. the order of the names in group_names does not match the inventory parent-child order; and

​There is no way of knowing that, as the inventory does not export that information to other ansible parts​

Ah, that’s a bummer. Is there any way to get the inventory to expose this information for lookup plugins to use?

  1. I don’t know how to prefix “group_files” to each group name, in order to keep my group files in their own directory.

perhaps I miss your point but

  • paths: “group_files/{{ group_names }}”
    files:
  • myfile

This can’t work. The “paths” attribute expects a list. By passing it “group_files/{{ group_names }}”, it turns into the string:

“group_files/[‘group1’, ‘group2’, ‘group3’]”

This is a weird, but valid path, so ansible will look in there, but not find anything.

4

I should have added that given the list a list [‘group1’, ‘group2’], I want to pass it to ansible as:

[‘group_files/group1’, ‘group_files/group2’]

5

Hi Serge,

1. the order of the names in group_names does not match the inventory
parent-child order; and

​There is no way of knowing that, as the inventory does not export that
information to other ansible parts​

Ah, that's a bummer. Is there any way to get the inventory to expose this
information for lookup plugins to use?

​No. There is nothing in the api that allows this without an update to th​e
core code.
It is though something to consider, when we have a look at the inventory
after 2.0.

2. I don't know how to prefix "group_files" to each group name, in order

to keep my group files in their own directory.

perhaps I miss your point but

  - paths: "group_files/{{ group_names }}"
    files:
            - myfile

This can't work. The "paths" attribute expects a list. By passing it
"group_files/{{ group_names }}", it turns into the string:

"group_files/['group1', 'group2', 'group3']"

This is a weird, but valid path, so ansible will look in there, but not
find anything.

​Yes, sorry, I missed that obviously.​

​Slow Saturday here :slight_smile:

This would require the possibility to redifine lists​. Allowing set_fact to
work with loops. Which is not possible either now, but will also hopefully
become a thing post 2.0

Serge

6

Hi Serge,

​No. There is nothing in the api that allows this without an update to th​e core code.

It is though something to consider, when we have a look at the inventory after 2.0.

Okay, thanks for confirming this.

The reason I have asked this question is that we’re using ansible in pull mode, with ansible running locally. This has the downside that all our roles, group_vars and host_vars are downloaded onto every host, thereby exposing some sensitive data to hosts that shouldn’t see it. Naturally, we cannot use the vault, because it requires password entry. And trying to figure out the roles that a host needs also appears undoable.

I have figured out a simple way to built rsync filters, that ensure that a pull-mode host only gets its own host_vars file, and group_vars files of the groups it belongs to. This has solved the issue of variables.

However, roles are a different story. We have several roles for different things, and many of these roles contain files that are only needed on one or a small subset of hosts. I would like to distribute these files only to the hosts that need them. This is the reason I asked my question.

For now, I have a work-around, that works, but is ugly. In my roles, wherever I need to copy a file, I’m doing this:

copy: content=‘{{ file_content }}’ dest=/path/to/dest

I’m keeping the entire file’s content in a variable called “file_content” in that host’s host_vars file, or in a group_vars file if appropriate. For lines of a few lines, this isn’t bad. But files that are bigger look ugly in a variable.

So if Ansible had the concept of host_files and group_files, with some kind of “first found” logic, it would be really nice.

Regards,
Anand

7

Hum…could you make use of git or svn, and just checkout host specific branches? Might lead to a bit a bloat on the server side, but would keep the client side cleaner.

Don

8

Hi Don,

Hum...could you make use of git or svn, and just checkout host specific

branches? Might lead to a bit a bloat on the server side, but would keep
the client side cleaner.

We floated this idea early on in our design, and our entire team felt that
a per-host branch would add too much overhead for us. Imagine managing a
few hundred hosts, and having to switch branches for each host to make
changes...

I gave this some more thought last night, and it turns out that many of the
files we distribute are scripts or secret-less config files, and it's not
too bad if they are exposed to all our hosts. The number of files that
contain actual secrets are few, and the secrets are things like licence
files, so my trick of keeping them in variables works well.

While Ansible was designed to be used in "push" mode, so that no state or
sensitive data needs to be kept on managed servers, it also has the "pull"
mode, which I really like, so we don't have to push anything. Clients check
in when they can and reconfigure themselves. Therefore, something that
would allow selective exposure of files and variables would be great!

Anand

9

Hi! Has anyone found any solution to this problem? Or a good workaround?
I’m hitting the same issue, that I would like to have host/group or inventory (like dev,prod) specific files, maybe also templates.
Storing them in variables would not work well, when e.g. you care about indentation, or when the data is binary…

Thanks for any help with that!
Krzysztof

W dniu niedziela, 19 kwietnia 2015 18:02:22 UTC+2 użytkownik Anand Buddhdev napisał: