# Secret variables / data: encryption **URL:** \ **Category:** Ansible Project\ **Created:** [June 29, 2013, 10:18pm UTC](https://forum.ansible.com/t/secret-variables-data-encryption/13042 "2013-06-29T22:18:30Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![jpmens1](https://avatars.discourse-cdn.com/v4/letter/j/e9c0ed/32.png) [@jpmens1](https://forum.ansible.com/u/jpmens1)\ **Post date:** [June 29, 2013, 10:18pm UTC](https://forum.ansible.com/t/secret-variables-data-encryption/13042/1 "2013-06-29T22:18:30Z")
I believe this topic has been requested before, and we talked about it at some length this afternoon in Antwerp. People wanting to store "confidential" variables with their playbooks, say, in a repo, cannot really do that for fear of exposing the secret variables. Would there be some interest in having support in Ansible for encrypted variables? Before investing a lot of time and then having the idea dropped, I'd like a bit of feedback on this. I see two possible ways we could go about implementing this. Take host\_vars/ for example, we could add what I'd like to call encrypted "pouches" of YAML which would be decrypted by Ansible when it attempts to access the pouch.         host\_vars/www normal YAML variables         host\_vars/mail.pouch encrypted YAML / base64-encoded Alternatively, a method proposed by Toshaan which I also rather like is mix encrypted variables into a regular vars file: ---
**Author:** ![Michael\_DeHaan2](https://avatars.discourse-cdn.com/v4/letter/m/da6949/32.png) [@Michael\_DeHaan2](https://forum.ansible.com/u/Michael_DeHaan2)\ **Post date:** [June 29, 2013, 11:51pm UTC](https://forum.ansible.com/t/secret-variables-data-encryption/13042/2 "2013-06-29T23:51:34Z")
This is very interesting… I like the ‘pouch’ idea, though I am not sure I like the name ‘pouch’ for it 🙂 The trick is you also want to decrypt it when editing the file, so it needs something for that too, that spawns $EDITOR around editing it, and Ansible needs to recognize if any of these are used to ask for unlock passwords, which sounds like it _MAY_ need to be another flag. Are people going to want to use different passwords for different files? The idea of handling it with a lookup plugin sounds a little promising as it’s not far off from the way the password plugin looks now for generating new passwords. (Is everybody aware of that?) Mostly I’m interested in how the user would interact with editing the file, and in what situations and when they are asked for a password. ---
**Author:** ![jpmens1](https://avatars.discourse-cdn.com/v4/letter/j/e9c0ed/32.png) [@jpmens1](https://forum.ansible.com/u/jpmens1)\ **Post date:** [June 30, 2013, 7:55am UTC](https://forum.ansible.com/t/secret-variables-data-encryption/13042/3 "2013-06-30T07:55:20Z")
> This is very interesting... I like the 'pouch' idea, though I am not sure I > like the name 'pouch' for it 🙂 Call it container or whatever -- I think a name that clearly identifies it as something special is good. Considering how much you like cows, maby 'udder' is what we need ... ? 😉 > Are people going to want to use different passwords for different files? I'm not thinking passwords (as in "type that in"), but rather key files (stored in a well-known path or specified as cfg variable/environment variable). We need something that can work unattended (think cron) and something which can be shared accross a team of people working on a distributed copy of the playbook repo. > The idea of handling it with a lookup plugin sounds a little promising as > it's not far off from the way the password plugin looks now for generating > new passwords. (Is everybody aware of that?) I wasn't, no. > Mostly I'm interested in how the user would interact with editing the file, > and in what situations and when they are asked for a password. I think a small utility which decrypts with said key file, invokes $EDITOR, and encrypts would suffice. On the other hand, as alluded to, people could write their own stuff using `openssl' cli.         -JP ---
**Author:** ![Vincent\_Van\_der\_Kuss](https://avatars.discourse-cdn.com/v4/letter/v/7c8e57/32.png) [@Vincent\_Van\_der\_Kuss](https://forum.ansible.com/u/Vincent_Van_der_Kuss)\ **Post date:** [June 30, 2013, 8:06am UTC](https://forum.ansible.com/t/secret-variables-data-encryption/13042/4 "2013-06-30T08:06:09Z")
> > This is very interesting... I like the 'pouch' idea, though I am not sure I > > like the name 'pouch' for it 🙂 > > Call it container or whatever -- I think a name that clearly identifies > it as something special is good. Considering how much you like cows, > maby 'udder' is what we need ... ? 😉 Just a suggestion here : vault > > Are people going to want to use different passwords for different files? > > I'm not thinking passwords (as in "type that in"), but rather key files > (stored in a well-known path or specified as cfg variable/environment > variable). > > We need something that can work unattended (think cron) and something > which can be shared accross a team of people working on a distributed > copy of the playbook repo. > > > The idea of handling it with a lookup plugin sounds a little promising as > > it's not far off from the way the password plugin looks now for generating > > new passwords. (Is everybody aware of that?) > > I wasn't, no. > > > Mostly I'm interested in how the user would interact with editing the file, > > and in what situations and when they are asked for a password. > > I think a small utility which decrypts with said key file, invokes > $EDITOR, and encrypts would suffice. On the other hand, as alluded to, > people could write their own stuff using `openssl' cli. As discussed yesterday, i think what Chef does comes close to what we want to achieve [http://docs.opscode.com/essentials\_data\_bags.html](http://docs.opscode.com/essentials_data_bags.html) Vincent ---
**Author:** ![Marco\_Vito\_Moscarito](https://avatars.discourse-cdn.com/v4/letter/m/97f17d/32.png) [@Marco\_Vito\_Moscarito](https://forum.ansible.com/u/Marco_Vito_Moscarito)\ **Post date:** [June 30, 2013, 10:43am UTC](https://forum.ansible.com/t/secret-variables-data-encryption/13042/5 "2013-06-30T10:43:42Z")
We actually use git+git-crypt to encript / decrypt files (SSL Cert, GPG Keys, ..) and use hashed variable for users password. Pros: - Work from now - Easy to implement - Choise encrypt system that we prefer Cons: - Only GIT compatible (don't work with SVN/BZR/...) - File is TOTALLY crypted (no just a single variable) JM2C ---
**Author:** ![Dave\_Cottlehuber](https://avatars.discourse-cdn.com/v4/letter/d/dfb087/32.png) [@Dave\_Cottlehuber](https://forum.ansible.com/u/Dave_Cottlehuber)\ **Post date:** [June 30, 2013, 1:29pm UTC](https://forum.ansible.com/t/secret-variables-data-encryption/13042/6 "2013-06-30T13:29:28Z")
Sounds interesting! Can you share how to do this? A blog post would be PURRfect. A+ Dave ---
**Author:** ![Michael\_DeHaan2](https://avatars.discourse-cdn.com/v4/letter/m/da6949/32.png) [@Michael\_DeHaan2](https://forum.ansible.com/u/Michael_DeHaan2)\ **Post date:** [June 30, 2013, 3:00pm UTC](https://forum.ansible.com/t/secret-variables-data-encryption/13042/7 "2013-06-30T15:00:31Z")
" knife data bag from file BAG\_NAME ITEM\_NAME.json" This is perhaps the worst CLI design I have ever seen 🙂 Perhaps someone can try to boil it down to something more simpler in how we think this would look at CLI level. ---
**Author:** ![jpmens1](https://avatars.discourse-cdn.com/v4/letter/j/e9c0ed/32.png) [@jpmens1](https://forum.ansible.com/u/jpmens1)\ **Post date:** [June 30, 2013, 5:51pm UTC](https://forum.ansible.com/t/secret-variables-data-encryption/13042/8 "2013-06-30T17:51:34Z")
> " knife data bag from file BAG\_NAME ITEM\_NAME.json" > > This is perhaps the worst CLI design I have ever seen 🙂 It \*is\* the worst I've ever seen. No worries -- we'll make that much better (which isn't hard to do)         -JP ---
**Author:** ![GW1](https://avatars.discourse-cdn.com/v4/letter/g/c89c15/32.png) [@GW1](https://forum.ansible.com/u/GW1)\ **Post date:** [June 30, 2013, 8:21pm UTC](https://forum.ansible.com/t/secret-variables-data-encryption/13042/9 "2013-06-30T20:21:48Z")
Hi, Why not use the ‘password’ lookup plugin (with\_password or {{ lookup(‘password’,…) }}) or ‘file’ lookup plugin? This way all confidential variables can be stored in a directory ‘credentials’ and either you do not commit them at all (the ones who have it could use it) or use .zip/.rar with password or encrypted loopback image and commit the encrypted archive? Greetings, gw ---
**Author:** ![Michael\_DeHaan](https://avatars.discourse-cdn.com/v4/letter/m/c4cdca/32.png) [@Michael\_DeHaan](https://forum.ansible.com/u/Michael_DeHaan)\ **Post date:** [June 30, 2013, 8:23pm UTC](https://forum.ansible.com/t/secret-variables-data-encryption/13042/10 "2013-06-30T20:23:05Z")
The password lookup plugin generates passwords when they are not already set, it is not a means of keeping confidental data that you can hand enter. The “vault” idea is a neat idea. ---
**Author:** ![Ernest0x](https://avatars.discourse-cdn.com/v4/letter/e/ed655f/32.png) [@Ernest0x](https://forum.ansible.com/u/Ernest0x)\ **Post date:** [June 30, 2013, 8:38pm UTC](https://forum.ansible.com/t/secret-variables-data-encryption/13042/11 "2013-06-30T20:38:56Z")
Another approach would be to have a ‘secret\_vars’ playbook section as a list with the names of variables whose values are expected to be in a format like this: shared\_key\_configuration\_name!base64.b64encode(encrypted\_value) For example, in playbooks: and in the inventory or in variable files : Then, multiple shared key configurations would normally be placed in the ansible configuration, like this: That way it would also be super easy to hide variables that are in secret\_vars list from ansible’s logs which is also a very important security enhancement measure. What do you think? ---
**Author:** ![Michael\_DeHaan2](https://avatars.discourse-cdn.com/v4/letter/m/da6949/32.png) [@Michael\_DeHaan2](https://forum.ansible.com/u/Michael_DeHaan2)\ **Post date:** [June 30, 2013, 10:08pm UTC](https://forum.ansible.com/t/secret-variables-data-encryption/13042/12 "2013-06-30T22:08:24Z")
I kind this much harder to explain than JP Men’s suggestion. We will be going with that 🙂 ---
**Author:** ![Michael\_DeHaan2](https://avatars.discourse-cdn.com/v4/letter/m/da6949/32.png) [@Michael\_DeHaan2](https://forum.ansible.com/u/Michael_DeHaan2)\ **Post date:** [June 30, 2013, 10:08pm UTC](https://forum.ansible.com/t/secret-variables-data-encryption/13042/13 "2013-06-30T22:08:47Z")
Ignore my inability to plac’e apostrophe’s correctly btw 🙂 ---
**Author:** ![Ernest0x](https://avatars.discourse-cdn.com/v4/letter/e/ed655f/32.png) [@Ernest0x](https://forum.ansible.com/u/Ernest0x)\ **Post date:** [June 30, 2013, 11:19pm UTC](https://forum.ansible.com/t/secret-variables-data-encryption/13042/14 "2013-06-30T23:19:23Z")
On second thought, to make things even simpler, ‘secret\_vars’ is not even needed. A ‘decrypt’ lookup plugin would be just enough to get the decrypted value of a variable in a format like the one I suggested. That way, the format can be made even simpler by moving away ‘shared\_key\_configuration\_name’ from it and giving it instead as a parameter to the ‘decrypt’ lookup plugin call. Pretty simple I think, without extra playbook syntax or additional, specially-handled files. And perhaps it has the easiest implementation since lookup logic is already available. ---
**Author:** ![Michael\_DeHaan](https://avatars.discourse-cdn.com/v4/letter/m/c4cdca/32.png) [@Michael\_DeHaan](https://forum.ansible.com/u/Michael_DeHaan)\ **Post date:** [June 30, 2013, 11:28pm UTC](https://forum.ansible.com/t/secret-variables-data-encryption/13042/15 "2013-06-30T23:28:22Z")
I generally am not a fan of lookup plugin syntax, especially when it has to return a data structure. It’s fine for some things, but the main intention of lookup plugins were to enable things like “with\_items” and then they got used for things like file lookups later, and the syntax is ok… but it’s something I find myself using rarely. I really like the vault idea because it would work transparently with vars\_files and other things, and _not_ require extra syntax. I am tentatively thinking ansible-vault --unlock ansible-vault --lock And then when ansible runs if a vault is used, it would prompt for the unlock password for that vault. The question is do you need multiple vault passwords at the same time or not. The other slight thing is you want to make sure none of your files get checked into git while locked down though, which might be a little tricky… I would hate to have to have people to know how to set up git triggers for this, but that might be the only way. ---
**Author:** ![Ernest0x](https://avatars.discourse-cdn.com/v4/letter/e/ed655f/32.png) [@Ernest0x](https://forum.ansible.com/u/Ernest0x)\ **Post date:** [July 1, 2013, 9:24am UTC](https://forum.ansible.com/t/secret-variables-data-encryption/13042/16 "2013-07-01T09:24:21Z")
In my opinion, a requirement to always prompt for a password does not fit with the purpose of automating things. Instead, there can be multiple vault configuration sections in the ansible configuration and ansible-vault could be called without a path (getting the paths from the configuration). For example: If ‘password\_file’ is set, it will be used to read the password for unlocking the vault. Only if it is not set would ansible utilities prompt to enter a password. Running an ansible binary would involve invoking ansible-vault to unlock the vaults in temporary paths (perhaps somewhere inside /tmp or ~/.ansible) with local user readonly access. The script would read vars from there, flag them internally as ‘sensitive’ so as not to leak them in logs, and, in the end, it would invoke again ansible-vault to remove these temporary paths. Password files could be shared among machines/users that run ansible. For even better security, they could be stored encrypted with the ssh public key of the local user and be decrypted before being used. ssh-agent could be used to avoid repeated password input prompts in the same fashion that it can be used to ssh to remote hosts. The ‘cipher’ setting could be used to support different encryption algorithms. If missing, a default one would be used. Editing vaults could be done with an ‘ansible-vault --edit {vault\_name | vault\_path}’ command that would unlock to a temporary path, spawn an editor and, on exit, encrypt the temporary path and move it to the permanent (locked) vault path. What do you mean by ‘make sure none of your files get checked into git while locked down’? Isn’t it the desirable behaviour to always check the files into GIT in a locked state? ---
**Author:** ![jpmens1](https://avatars.discourse-cdn.com/v4/letter/j/e9c0ed/32.png) [@jpmens1](https://forum.ansible.com/u/jpmens1)\ **Post date:** [July 1, 2013, 9:36am UTC](https://forum.ansible.com/t/secret-variables-data-encryption/13042/17 "2013-07-01T09:36:19Z")
Thanks for all your comments and ideas. I have submitted a pull-request [1] which implements this as a filter. I've tried to make it so that people can use as many keys as they desire. The page with the PR [1] contains a blurb about how to get started and how to use it. A utility called ansible-vault is included. Regards,         -JP [1] [https://github.com/ansible/ansible/pull/3393](https://github.com/ansible/ansible/pull/3393) ---
**Author:** ![jpmens1](https://avatars.discourse-cdn.com/v4/letter/j/e9c0ed/32.png) [@jpmens1](https://forum.ansible.com/u/jpmens1)\ **Post date:** [July 1, 2013, 9:37am UTC](https://forum.ansible.com/t/secret-variables-data-encryption/13042/18 "2013-07-01T09:37:12Z")
> In my opinion, a requirement to always prompt for a password does not > fit with the purpose of automating things. Full ACK         -JP ---
**Author:** ![jpmens1](https://avatars.discourse-cdn.com/v4/letter/j/e9c0ed/32.png) [@jpmens1](https://forum.ansible.com/u/jpmens1)\ **Post date:** [July 1, 2013, 9:39am UTC](https://forum.ansible.com/t/secret-variables-data-encryption/13042/19 "2013-07-01T09:39:13Z")
> If 'password\_file' is set, it will be used to read the password for > unlocking the vault. Only if it is not set would ansible utilities > prompt to enter a password. I'm not understanding the whole lock/unlock thing. IMO (and that is how I've done it) encrypted values are contained in vars files and are decrypted on use.         -JP ---
**Author:** ![Ernest0x](https://avatars.discourse-cdn.com/v4/letter/e/ed655f/32.png) [@Ernest0x](https://forum.ansible.com/u/Ernest0x)\ **Post date:** [July 1, 2013, 12:44pm UTC](https://forum.ansible.com/t/secret-variables-data-encryption/13042/20 "2013-07-01T12:44:41Z")
Correct. Encrypting / Decrypting are better words than Locking / Unlocking to describe the usage I suggested previously (decrypting vault files somewhere temporarily while vars are being used), but I used the latter coming from a discussion with references to that terminology. Also, although I suggested using temporary files for the decrypted vaults earlier, on second thought, I think that they should be better kept temporarily in memory. That way you can be sure that no decrypted values are left over on unpredicted conditions (e.g. on crashes). [Next page](https://forum.ansible.com/t/secret-variables-data-encryption/13042.md?page=2)