Reading in extra files, as or into dicts?

Hi all,

I have created a directory 'users' alongside my inventory. It has a directory 'user_vars', intended to be used like host_vars, but for users, obviously.

In there, I have files like this:

I've got this, but it looks horrible:

I have created a directory 'users' alongside my inventory. It has a directory 'user_vars', intended to be used like host_vars, but for users, obviously.

I've got this, but it looks horrible:

==================
- name: set up user dicts
set_fact:

You could avoid this by just using regular group_vars/.

Are there ways to make this cleaner?

Probably not the answer you're looking for: While I use ansible for many things I'd always prefer a decent user management system over hacking this into ansible playbooks. Maintaining users, groups and maybe sudoers in a database gives you required features like enforcing unique constraints (username, POSIX-UID/-GID), data references, audit reports etc. There are various ready-to-use systems (most times based on some LDAP server) to choose from.

Ciao, Michael.

Any suggestions?

Questions more than suggestions, but you can take them as such anyway.

  1. How many users are we talking about? If the user data is smaller than the code it takes to manipulate it, maybe it’d be better to make the data more complex and the code simpler.

  2. Are these files the canonical source of truth, or are they downstream? If downstream, perhaps it would be better to pull the data from upstream as needed. If these files are canonical truth, then either restructure them in a way that supports cleaner access, or consider creating an upstream data source. The latter would only make sense if the answer to #1 was large enough to justify it.

If it were me, and assuming a fairly small number of users and/or relatively static data (i.e. I’m going to maintain this by hand for the foreseeable future), I’d put all these user data into defaults/main.yml of a custom “my_users” role, probably in a single dict keyed by ID, or maybe a list. Make everything that might have multiple values for any user a list - ssh keys for example - so even if a user has only one (or none) you access that data the same way for everybody.

The fact that the expressions you’re having to use to access the data currently are unwieldy indicates the initial data structures are a problem. Having user data split into multiple files may seem like a simplification, but I think it’s just the opposite.

Good luck!

For example, given the tree

tree inventory/

inventory/
├── group_vars
│ └── all
├── hosts
└── users
    ├── public_keys
    │ ├── alice@bar
    │ ├── alice@foo
    │ ├── richard@bar
    │ └── richard@foo
    └── user_vars
        ├── alice
        └── richard

and the files for testing

cat inventory/users/user_vars/alice

name: alice
gecos: 'Alice Beatrice,'
shell: '/bin/bash'
ssh_keys:
   - alice@foo
   - alice@bar

cat inventory/users/user_vars/richard

name: richard
gecos: 'Richard Hector,'
shell: '/bin/bash'
ssh_keys:
   - richard@foo
   - richard@bar

cat inventory/users/public_keys/

alice@bar alice@foo richard@bar richard@foo

cat inventory/users/public_keys/alice@bar

ssh-rsa djflskdjflsdkjflsdjflkdjl alice@bar

cat inventory/users/public_keys/alice@foo

ssh-rsa djflskdjflsdkjflsdjflkdjl alice@foo

cat inventory/users/public_keys/richard@bar

ssh-rsa djflskdjflsdkjflsdjflkdjl richard@bar

cat inventory/users/public_keys/richard@foo

ssh-rsa djflskdjflsdkjflsdjflkdjl richard@foo

* Create variables

  inventory_dir: inventory
  dyn_vars_list: [user_vars, public_keys]

* Get the names of the files

    - set_fact:
        keys: "{{ keys|d({})|
                  combine({item: files}) }}"
      loop: "{{ dyn_vars_list }}"
      vars:
        fileglob: "{{ inventory_dir }}/users/{{ item }}/*"
        files: "{{ query('fileglob', fileglob)|
                   map('basename')|list }}"

gives

  keys:
    public_keys:
    - richard@bar
    - alice@bar
    - alice@foo
    - richard@foo
    user_vars:
    - alice
    - richard

* Get the content of the files

    - set_fact:
        values: "{{ values|d() +
                    [{'var': item.0.key,
                      'key': item.1,
                      'value': value}] }}"
      with_subelements:
        - "{{ keys|dict2items }}"
        - value
      vars:
        path: "{{ inventory_dir }}/users/{{ item.0.key }}/{{ item.1 }}"
        value: "{{ lookup('file', path)|from_yaml }}"

gives

  values:
  - key: alice
    value:
      gecos: Alice Beatrice,
      name: alice
      shell: /bin/bash
      ssh_keys:
      - alice@foo
      - alice@bar
    var: user_vars
  - key: richard
    value:
      gecos: Richard Hector,
      name: richard
      shell: /bin/bash
      ssh_keys:
      - richard@foo
      - richard@bar
    var: user_vars
  - key: richard@bar
    value: ssh-rsa dkjfslkdjflskdjflsdkjflsdjflkdjl richard@bar
    var: public_keys
  - key: alice@bar
    value: ssh-rsa dkjfslkdjflskdjflsdkjflsdjflkdjl alice@bar
    var: public_keys
  - key: alice@foo
    value: ssh-rsa dkjfslkdjflskdjflsdkjflsdjflkdjl alice@foo
    var: public_keys
  - key: richard@foo
    value: ssh-rsa dkjfslkdjflskdjflsdkjflsdjflkdjl richard@foo
    var: public_keys

* Group the values by 'var' and create the dictionary

    - set_fact:
        dyn_vars: "{{ dyn_vars|d({})|
                      combine({item.0: item.1|
                               items2dict(key_name='key',
                                          value_name='value')}) }}"
      loop: "{{ values|groupby('var') }}"

gives

  dyn_vars:
    public_keys:
      alice@bar: ssh-rsa djflskdjflsdkjflsdjflkdjl alice@bar
      alice@foo: ssh-rsa djflskdjflsdkjflsdjflkdjl alice@foo
      richard@bar: ssh-rsa djflskdjflsdkjflsdjflkdjl richard@bar
      richard@foo: ssh-rsa djflskdjflsdkjflsdjflkdjl richard@foo
    user_vars:
      alice:
        gecos: Alice Beatrice,
        name: alice
        shell: /bin/bash
        ssh_keys:
        - alice@foo
        - alice@bar
      richard:
        gecos: Richard Hector,
        name: richard
        shell: /bin/bash
        ssh_keys:
        - richard@foo
        - richard@bar

The usage of the dictionary should be trivial.