I believe this topic has been requested before, and we talked about it
at some length this afternoon in Antwerp.
People wanting to store "confidential" variables with their playbooks,
say, in a repo, cannot really do that for fear of exposing the secret
variables.
Would there be some interest in having support in Ansible for encrypted
variables? Before investing a lot of time and then having the idea
dropped, I'd like a bit of feedback on this.
I see two possible ways we could go about implementing this. Take
host_vars/ for example, we could add what I'd like to call encrypted
"pouches" of YAML which would be decrypted by Ansible when it attempts
to access the pouch.
host_vars/www normal YAML variables
host_vars/mail.pouch encrypted YAML / base64-encoded
Alternatively, a method proposed by Toshaan which I also rather like is
mix encrypted variables into a regular vars file:
This is very interesting… I like the ‘pouch’ idea, though I am not sure I like the name ‘pouch’ for it
The trick is you also want to decrypt it when editing the file, so it needs something for that too, that spawns $EDITOR around editing it, and Ansible needs to recognize if any of these are used to ask for unlock passwords, which sounds like it MAY need to be another flag.
Are people going to want to use different passwords for different files?
The idea of handling it with a lookup plugin sounds a little promising as it’s not far off from the way the password plugin looks now for generating new passwords. (Is everybody aware of that?)
Mostly I’m interested in how the user would interact with editing the file, and in what situations and when they are asked for a password.
This is very interesting... I like the 'pouch' idea, though I am not sure I
like the name 'pouch' for it
Call it container or whatever -- I think a name that clearly identifies
it as something special is good. Considering how much you like cows,
maby 'udder' is what we need ... ?
Are people going to want to use different passwords for different files?
I'm not thinking passwords (as in "type that in"), but rather key files
(stored in a well-known path or specified as cfg variable/environment
variable).
We need something that can work unattended (think cron) and something
which can be shared accross a team of people working on a distributed
copy of the playbook repo.
The idea of handling it with a lookup plugin sounds a little promising as
it's not far off from the way the password plugin looks now for generating
new passwords. (Is everybody aware of that?)
I wasn't, no.
Mostly I'm interested in how the user would interact with editing the file,
and in what situations and when they are asked for a password.
I think a small utility which decrypts with said key file, invokes
$EDITOR, and encrypts would suffice. On the other hand, as alluded to,
people could write their own stuff using `openssl' cli.
This is very interesting... I like the 'pouch' idea, though I am not sure I
like the name 'pouch' for it
Call it container or whatever -- I think a name that clearly identifies
it as something special is good. Considering how much you like cows,
maby 'udder' is what we need ... ?
Just a suggestion here : vault
Are people going to want to use different passwords for different files?
I'm not thinking passwords (as in "type that in"), but rather key files
(stored in a well-known path or specified as cfg variable/environment
variable).
We need something that can work unattended (think cron) and something
which can be shared accross a team of people working on a distributed
copy of the playbook repo.
The idea of handling it with a lookup plugin sounds a little promising as
it's not far off from the way the password plugin looks now for generating
new passwords. (Is everybody aware of that?)
I wasn't, no.
Mostly I'm interested in how the user would interact with editing the file,
and in what situations and when they are asked for a password.
I think a small utility which decrypts with said key file, invokes
$EDITOR, and encrypts would suffice. On the other hand, as alluded to,
people could write their own stuff using `openssl' cli.
As discussed yesterday, i think what Chef does comes close to what we
want to achieve
Why not use the ‘password’ lookup plugin (with_password or {{ lookup(‘password’,…) }}) or ‘file’ lookup plugin? This way all confidential variables can be stored in a directory ‘credentials’ and either you do not commit them at all (the ones who have it could use it) or use .zip/.rar with password or encrypted loopback image and commit the encrypted archive?
Another approach would be to have a ‘secret_vars’ playbook section as a list with the names of variables whose values are expected to be in a format like this: shared_key_configuration_name!base64.b64encode(encrypted_value) For example, in playbooks: and in the inventory or in variable files : Then, multiple shared key configurations would normally be placed in the ansible configuration, like this: That way it would also be super easy to hide variables that are in secret_vars list from ansible’s logs which is also a very important security enhancement measure. What do you think?
On second thought, to make things even simpler, ‘secret_vars’ is not even needed. A ‘decrypt’ lookup plugin would be just enough to get the decrypted value of a variable in a format like the one I suggested. That way, the format can be made even simpler by moving away ‘shared_key_configuration_name’ from it and giving it instead as a parameter to the ‘decrypt’ lookup plugin call. Pretty simple I think, without extra playbook syntax or additional, specially-handled files. And perhaps it has the easiest implementation since lookup logic is already available.
I generally am not a fan of lookup plugin syntax, especially when it has to return a data structure. It’s fine for some things, but the main intention of lookup plugins were to enable things like “with_items” and then they got used for things like file lookups later, and the syntax is ok… but it’s something I find myself using rarely.
I really like the vault idea because it would work transparently with vars_files and other things, and not require extra syntax.
I am tentatively thinking
ansible-vault --unlock
ansible-vault --lock
And then when ansible runs if a vault is used, it would prompt for the unlock password for that vault.
The question is do you need multiple vault passwords at the same time or not.
The other slight thing is you want to make sure none of your files get checked into git while locked down though, which might be a little tricky… I would hate to have to have people to know how to set up git triggers for this, but that might be the only way.
In my opinion, a requirement to always prompt for a password does not fit with the purpose of automating things. Instead, there can be multiple vault configuration sections in the ansible configuration and ansible-vault could be called without a path (getting the paths from the configuration). For example: If ‘password_file’ is set, it will be used to read the password for unlocking the vault. Only if it is not set would ansible utilities prompt to enter a password. Running an ansible binary would involve invoking ansible-vault to unlock the vaults in temporary paths (perhaps somewhere inside /tmp or ~/.ansible) with local user readonly access. The script would read vars from there, flag them internally as ‘sensitive’ so as not to leak them in logs, and, in the end, it would invoke again ansible-vault to remove these temporary paths. Password files could be shared among machines/users that run ansible. For even better security, they could be stored encrypted with the ssh public key of the local user and be decrypted before being used. ssh-agent could be used to avoid repeated password input prompts in the same fashion that it can be used to ssh to remote hosts. The ‘cipher’ setting could be used to support different encryption algorithms. If missing, a default one would be used. Editing vaults could be done with an ‘ansible-vault --edit {vault_name | vault_path}’ command that would unlock to a temporary path, spawn an editor and, on exit, encrypt the temporary path and move it to the permanent (locked) vault path. What do you mean by ‘make sure none of your files get checked into git while locked down’? Isn’t it the desirable behaviour to always check the files into GIT in a locked state?
Thanks for all your comments and ideas. I have submitted a pull-request
[1] which implements this as a filter. I've tried to make it so that
people can use as many keys as they desire. The page with the PR [1]
contains a blurb about how to get started and how to use it.
If 'password_file' is set, it will be used to read the password for
unlocking the vault. Only if it is not set would ansible utilities
prompt to enter a password.
I'm not understanding the whole lock/unlock thing. IMO (and that is how
I've done it) encrypted values are contained in vars files and are
decrypted on use.
Correct. Encrypting / Decrypting are better words than Locking /
Unlocking to describe the usage I suggested previously (decrypting vault
files somewhere temporarily while vars are being used), but I used the
latter coming from a discussion with references to that terminology.
Also, although I suggested using temporary files for the decrypted
vaults earlier, on second thought, I think that they should be better
kept temporarily in memory. That way you can be sure that no decrypted
values are left over on unpredicted conditions (e.g. on crashes).
