# Secret variables / data: encryption

**URL:** <https://forum.ansible.com/t/secret-variables-data-encryption/13042>\
**Category:** Ansible Project\
**Created:** [June 29, 2013, 10:18pm UTC](https://forum.ansible.com/t/secret-variables-data-encryption/13042 "2013-06-29T22:18:30Z")\
**Posts on this page:** 20\
**Page:** 3

<div class="post-metadata">

**Author:** ![Ernest0x](https://avatars.discourse-cdn.com/v4/letter/e/ed655f/32.png) [@Ernest0x](https://forum.ansible.com/u/Ernest0x)\
**Post date:** [July 2, 2013, 1:56pm UTC](https://forum.ansible.com/t/secret-variables-data-encryption/13042/41 "2013-07-02T13:56:06Z")

</div>

I like the “vault:” prefix idea. Maybe it can also be used when editing files through ansible-vault, so that, before spawning the editor, all ‘’ values are translated to a ready-to-edit ‘’ expression. For variables encrypted with a key other than the default, it could try decrypting the value with each available key until it succeeds so as to produce a correct ‘’ expression. Another option to avoid that loop (but adding some more text in the ecrypted-value syntax) would be to use a vault+keyname combination prefix. For example,

---

<div class="post-metadata">

**Author:** ![jpmens1](https://avatars.discourse-cdn.com/v4/letter/j/e9c0ed/32.png) [@jpmens1](https://forum.ansible.com/u/jpmens1)\
**Post date:** [July 2, 2013, 1:57pm UTC](https://forum.ansible.com/t/secret-variables-data-encryption/13042/42 "2013-07-02T13:57:03Z")

</div>

> I know this is a BIG ask, but for environments with multiple Admins an  
> implementation of Shamir's secret would rock.

How would that work? Say you have 3 admins, all must be present (and  
feed a third of a key) when running a playbook? Hmm. And how do they  
cron' it?

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;-JP

---

<div class="post-metadata">

**Author:** ![Jeroen\_Hoekx](https://avatars.discourse-cdn.com/v4/letter/j/ba9def/32.png) [@Jeroen\_Hoekx](https://forum.ansible.com/u/Jeroen_Hoekx)\
**Post date:** [July 2, 2013, 1:57pm UTC](https://forum.ansible.com/t/secret-variables-data-encryption/13042/43 "2013-07-02T13:57:32Z")

</div>

Interesting. How would you see that in practice?

The scenario I can imagine right now is that you need two keys to  
decrypt the secret. One could be stored on the management server and  
the other should be provided by the sysadmin? That way password  
sharing between people is not needed.

(My masters thesis was about this and I've been looking for a  
practical use of it)

Jeroen

---

<div class="post-metadata">

**Author:** ![Brian\_Coca1](https://avatars.discourse-cdn.com/v4/letter/b/278dde/32.png) [@Brian\_Coca1](https://forum.ansible.com/u/Brian_Coca1)\
**Post date:** [July 2, 2013, 2:01pm UTC](https://forum.ansible.com/t/secret-variables-data-encryption/13042/44 "2013-07-02T14:01:17Z")

</div>

Shamir’s secret works as n/m, where n = min number of keys to decrypt and m = total number of keys.

No cron in this case. You don’t need all admin, you can define n of them, it can be 1. This way you can have a key per admin and have a ‘quorum’ defined to how many of them need access to decrypt.

I’ve used this in the past to store highly sensitive material (hardware root keys, enterprise key that decrypts everyone’s email, etc).

---

<div class="post-metadata">

**Author:** ![Lorin\_Hochstein1](https://avatars.discourse-cdn.com/v4/letter/l/ce73a5/32.png) [@Lorin\_Hochstein1](https://forum.ansible.com/u/Lorin_Hochstein1)\
**Post date:** [July 2, 2013, 2:05pm UTC](https://forum.ansible.com/t/secret-variables-data-encryption/13042/45 "2013-07-02T14:05:45Z")

</div>

I understand the motivation behind putting the secrets into the playbook  
and then converting them with an encrypting preprocessor, since it  
increases usability. But I'd be nervous about somebody accidentally  
committing the file to a version control system before running the  
preprocessor.

Lorin

---

<div class="post-metadata">

**Author:** ![jpmens1](https://avatars.discourse-cdn.com/v4/letter/j/e9c0ed/32.png) [@jpmens1](https://forum.ansible.com/u/jpmens1)\
**Post date:** [July 2, 2013, 2:10pm UTC](https://forum.ansible.com/t/secret-variables-data-encryption/13042/46 "2013-07-02T14:10:56Z")

</div>

> I understand the motivation behind putting the secrets into the playbook  
> and then converting them with an encrypting preprocessor, since it  
> increases usability. But I'd be nervous about somebody accidentally  
> committing the file to a version control system before running the  
> preprocessor.

We're talking of an 'editing' front-end, so this cannot (famous last  
words) really happen. A human uses `ansible-vault --edit' and is dropped  
into an editor. If said human adds secrets without wrapping them in  
a 'magic' token, then yes, it will happen. Otherwise, secrets are  
encrypted on the way out of the editor.

Anyway -- this is premature until the basics are settled (see my Recap  
message 🙂

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;-JP

---

<div class="post-metadata">

**Author:** ![Brian\_Coca1](https://avatars.discourse-cdn.com/v4/letter/b/278dde/32.png) [@Brian\_Coca1](https://forum.ansible.com/u/Brian_Coca1)\
**Post date:** [July 2, 2013, 5:17pm UTC](https://forum.ansible.com/t/secret-variables-data-encryption/13042/47 "2013-07-02T17:17:15Z")

</div>

I don’t really know how to apply this to an interactive app, I’ve mostly used for offline storage and in an emergency get the quorum needed to give access to the ‘highly sensitive secure material’

---

<div class="post-metadata">

**Author:** ![Marco\_Vito\_Moscarito](https://avatars.discourse-cdn.com/v4/letter/m/97f17d/32.png) [@Marco\_Vito\_Moscarito](https://forum.ansible.com/u/Marco_Vito_Moscarito)\
**Post date:** [July 2, 2013, 5:19pm UTC](https://forum.ansible.com/t/secret-variables-data-encryption/13042/48 "2013-07-02T17:19:32Z")

</div>

This is a snippet of what I use:

[https://coderwall.com/p/kucyaw](https://coderwall.com/p/kucyaw)

Let me know if ther's some info that are missing or if someone have  
better ideas 🙂

Marco

---

<div class="post-metadata">

**Author:** ![Dylan\_Martin](https://avatars.discourse-cdn.com/v4/letter/d/8797f3/32.png) [@Dylan\_Martin](https://forum.ansible.com/u/Dylan_Martin)\
**Post date:** [July 2, 2013, 7:58pm UTC](https://forum.ansible.com/t/secret-variables-data-encryption/13042/49 "2013-07-02T19:58:49Z")

</div>

Sorry for coming in late, and sorry again if I’m missing the point or just generally being stupid and oafish.

Unattended decryption is tricky because a lot of the time you wind up with something that’s no better than having your secrets saved in clear-text.

I can think of three ways this gets handled:

1. You have a password or key that gets loaded and used. The problem here is there are few scenarios where a baddie has access to the encrypted data and not the key. You can keep the key outside your git repo (or otherwise separated) but that isn’t any more secure than keeping the plain text of whatever you want to encrypt separate. EG you encrypt your db password and store it in your git repo and you use your keyczar key to decrypt the db password and access the db. Which is more complex and no more secure than storing your db password wherever the keyczar key was kept.

2. You keep plaintext data outside your version control. This offers protection when baddies getting ahold of your version control stuff, but does not protect you if they get access to your account or your data beyond the version control stuff (eg they get a complete copy of your home dir somehow and you kept that data in your home dir) This is roughly just as secure as option 1 and a whole lot simpler. I use this sometimes, where I’ll have a file called .pass or something inside my working copy of a repo and I’ll add .pass to the .gitignore file.

3. You have an agent, like ssh-agent, that holds the password. Said another way, you type one password into an agent and it makes a lot of encrypted stuff available to processes that can interact with it. If a baddie hacks your account, they can submit encrypted stuff to the agent, but It does offer protection in a case where the baddie gets a copy of your data, including the keys, but does not have access to your account. You could also use an agent that handles the encryption and produces clear text data (like your db password). This is better than 1 and 2 because it only fails if your account is hacked and baddies can use it.

In conclusion, if you do add something to handle stored encrypted data, we should make sure it’s actually more secure than just keeping the data in plain text in another directory. Using an agent might be one way to do that. There might be others.

Sorry if I’m not making sense or saying something stupid or obvious.

-Dylan

---

<div class="post-metadata">

**Author:** ![Michael\_DeHaan2](https://avatars.discourse-cdn.com/v4/letter/m/da6949/32.png) [@Michael\_DeHaan2](https://forum.ansible.com/u/Michael_DeHaan2)\
**Post date:** [July 2, 2013, 8:17pm UTC](https://forum.ansible.com/t/secret-variables-data-encryption/13042/50 "2013-07-02T20:17:03Z")

</div>

I’m very much in favor of the agent approach.

If there’s an unlock “password” equivalent in the Ansible tree and the files are just obfuscated, we need to avoid that.

if the agent type approach involves just having Ansible ask for a password, that’s ok… and I do like the idea of it spawning an editor to allow you to edit your password file in a transactional manner.

It is important that this need not rely on templates and lookup plugins and you can use regular vars\_files type things, or if absolutely neccessary, something _like_ vars\_files, like a vault\_files.

---

<div class="post-metadata">

**Author:** ![Ernest0x](https://avatars.discourse-cdn.com/v4/letter/e/ed655f/32.png) [@Ernest0x](https://forum.ansible.com/u/Ernest0x)\
**Post date:** [July 3, 2013, 1:33pm UTC](https://forum.ansible.com/t/secret-variables-data-encryption/13042/51 "2013-07-03T13:33:13Z")

</div>

An agent-based solution does not make your global security necessarily  
better because you still have a password, and that password should be  
somewhere stored securely. Actually, you may add an extra security  
measure on the systems on which you run ansible, but you should get more  
concerned about the security of the systems that keep your passwords.

That said, the use of keyczar that Jen-Piet Mens introduced would work  
fine with an agent-based password-protected session on a machine that  
stores the keys. For example, you can put your keys on a (hardened) host  
and access them from each managing machine through sshfs. Or you could  
perhaps put the keys in a usb stick within an encrypted filesystem that  
prompts for a password and gets automounted on the insertion of the  
device into the managing hosts.

So, I think what we should discuss is not moving away from keyczar, but  
what we can do to password-protect keyczar key stores and use them in an  
agent-based fashion. Also see how ansible-vault can help us in  
automating the involved processes.

---

<div class="post-metadata">

**Author:** ![Michael\_DeHaan2](https://avatars.discourse-cdn.com/v4/letter/m/da6949/32.png) [@Michael\_DeHaan2](https://forum.ansible.com/u/Michael_DeHaan2)\
**Post date:** [July 3, 2013, 1:39pm UTC](https://forum.ansible.com/t/secret-variables-data-encryption/13042/52 "2013-07-03T13:39:05Z")

</div>

If I had a password to unlock my secret vault I wouldn’t store it.

I think sshfs is a pretty ‘bolted on’ sort of thing (fuse?) and would like to avoid that.

What we need to do here is not discuss implementation and propose an implementation before we first arrive at a user interface and a specification.

I don’t want to sound too much like I’m trying to dictate what goes on here, but as I am the one who gets the brunt of all the user questions and needs to support things, I do care very much that it presents as little questions as possible about how it works, and it works as smoothly as possible.

I need to dig through JP’s code yet and see what I think about that approach and interface, and I’ll share some comments here later.

It obviously does not eliminate the possibility of folks adding in their own modules and plugins and things outside of core, but if we do have something like this in core, the interface at which it presents itself is very important to me.

---

<div class="post-metadata">

**Author:** ![Ernest0x](https://avatars.discourse-cdn.com/v4/letter/e/ed655f/32.png) [@Ernest0x](https://forum.ansible.com/u/Ernest0x)\
**Post date:** [July 3, 2013, 2:47pm UTC](https://forum.ansible.com/t/secret-variables-data-encryption/13042/53 "2013-07-03T14:47:25Z")

</div>

See Jan-Piet Mens' recap message. Currently, it is the most well-shaped  
approach which includes a suggested user interface and a discussion on  
how it can be further improved. Of course it's ok to not like that idea,  
but it would be better if there were alternative well-described  
suggestions or specification drafts. In fact, I believe that this is  
good time for this topic to be given a higher priority so that 1.3 be  
the first version of Ansible that does offer core options for using it  
with less fear in organizations with demanding security policies.

---

<div class="post-metadata">

**Author:** ![Michael\_DeHaan2](https://avatars.discourse-cdn.com/v4/letter/m/da6949/32.png) [@Michael\_DeHaan2](https://forum.ansible.com/u/Michael_DeHaan2)\
**Post date:** [July 3, 2013, 3:11pm UTC](https://forum.ansible.com/t/secret-variables-data-encryption/13042/54 "2013-07-03T15:11:17Z")

</div>

I’m interested, but give me some time on this.

Smiting the bug queue down to 0 needs to happen first.

---

<div class="post-metadata">

**Author:** ![jpmens1](https://avatars.discourse-cdn.com/v4/letter/j/e9c0ed/32.png) [@jpmens1](https://forum.ansible.com/u/jpmens1)\
**Post date:** [July 4, 2013, 7:50am UTC](https://forum.ansible.com/t/secret-variables-data-encryption/13042/55 "2013-07-04T07:50:24Z")

</div>

This topic is keeping me awake BIG TIME 🙂 I'd really like to see  
encrypted vars in Ansible, and the simple way of doing it [1] appears to  
be too simple; people want an agent.

How does the following sound?

A vault agent runs on a (protected) box on the network to which the  
Ansible manager has access to (this need not be a dedicated box:  
localhost will do very nicely). It's an RPC server which is responsible  
for key creation and storage in files, and its main task is  
encryption/decryption. Basically it does two things:

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;vault-agent --createkey mykey # create secret key & store it  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;vault-agent --run # start RPC server

On the Ansible side (ie within Ansible), we allow for \*.vault files in  
host\_vars/ and group\_vars/. These are YAML files which can only be  
created/edited with a special command:

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;ansible-vault --key mykey --file host\_vars/localhost.vault

The program (ansible-vault) checks with the RPC server whether 'mykey'  
exists and drops the user into $EDITOR. Upon exiting $EDITOR,  
ansible-vault sends the payload (i.e. what the human edited) to the  
vault-agent for encryption with the specified key and obtains the  
base64-encded result which is stored in host\_vars/localhost.vault as  
follows:

---

<div class="post-metadata">

**Author:** ![shirou](https://avatars.discourse-cdn.com/v4/letter/s/ea5d25/32.png) [@shirou](https://forum.ansible.com/u/shirou)\
**Post date:** [July 4, 2013, 8:38am UTC](https://forum.ansible.com/t/secret-variables-data-encryption/13042/56 "2013-07-04T08:38:47Z")

</div>

Hello,

I am not a user just know the name, but how about using gpg-agent (not  
ssh-agent)?

I glanced the python-gnupg doc[1], it can handle a gpg-agent with  
use\_agent argument.

(Since I haven't read all discussion, apologize if someone already  
discuss or I miss the point.)

[1] [http://pythonhosted.org/python-gnupg/](http://pythonhosted.org/python-gnupg/)

Regards,  
Shirou

---

<div class="post-metadata">

**Author:** ![jpmens1](https://avatars.discourse-cdn.com/v4/letter/j/e9c0ed/32.png) [@jpmens1](https://forum.ansible.com/u/jpmens1)\
**Post date:** [July 4, 2013, 8:50am UTC](https://forum.ansible.com/t/secret-variables-data-encryption/13042/57 "2013-07-04T08:50:12Z")

</div>

> I glanced the python-gnupg doc[1], it can handle a gpg-agent with  
> use\_agent argument.

That would mean people need a GPG infrastructure, which not everybody  
has ...

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;-JP

---

<div class="post-metadata">

**Author:** ![shirou](https://avatars.discourse-cdn.com/v4/letter/s/ea5d25/32.png) [@shirou](https://forum.ansible.com/u/shirou)\
**Post date:** [July 4, 2013, 8:57am UTC](https://forum.ansible.com/t/secret-variables-data-encryption/13042/58 "2013-07-04T08:57:14Z")

</div>

> \> I glanced the python-gnupg doc[1], it can handle a gpg-agent with  
> \> use\_agent argument.  
> That would mean people need a GPG infrastructure, which not everybody has ...

yah. that is a problem...

---

<div class="post-metadata">

**Author:** ![Michael\_DeHaan2](https://avatars.discourse-cdn.com/v4/letter/m/da6949/32.png) [@Michael\_DeHaan2](https://forum.ansible.com/u/Michael_DeHaan2)\
**Post date:** [July 4, 2013, 12:12pm UTC](https://forum.ansible.com/t/secret-variables-data-encryption/13042/59 "2013-07-04T12:12:13Z")

</div>

Here’s an interface I’d like. (Meaning, one I’d be happy to accept). Small variations fine of course but notice the degree of encapsulation I am looking for.

First off, it works on any file Ansible can read but the file has a header so Ansible can yell if it’s not running in a way that could decrypt them. There must be absolutely no special syntax in YAML files, no lookup plugins, no templates.

This would work with vars\_files, playbooks, whatever your want.

ansible-vault agent   
ansible-vault create # spawns $EDITOR but on save will encrpt file  
ansible-vault edit # spawns $EDITOR … if the file is unencrypted, just edit it like normal YAML, otherwise edit as encrypted file, on save, re-encrypts. Be careful of temp files of course  
ansible-playbook … # works fine because we started the vault agent, but transparently works on all files as unencrypted. Magic code lives in utils.parse\_yaml\_from\_file.

Sometimes you might want to re-encrypt things so you can do:

ansible-vault change-password # prompts for old and new password to change the password on the file and re-encrypt it, and works optionally on entire subdirectories, so you could change multiple locked YAML files all in one pass

- hosts: blarg  
vars\_files:  
…

Running ansible without ansible-vault first would produce an error “vault file used without agent connection, see ansible-vault documentation for details on how to access encrypted files” when you tried to load a file and it couldn’t decrypt it.

Finally you can shell exit to leave the agent, just like ssh-agent.

---

<div class="post-metadata">

**Author:** ![Ernest0x](https://avatars.discourse-cdn.com/v4/letter/e/ed655f/32.png) [@Ernest0x](https://forum.ansible.com/u/Ernest0x)\
**Post date:** [July 4, 2013, 2:37pm UTC](https://forum.ansible.com/t/secret-variables-data-encryption/13042/60 "2013-07-04T14:37:25Z")

</div>

First, I would like to make a comment that has more to do with the data representation and less with the UI: Both of the suggested interfaces are encrypting files as a whole, which destroys yaml structure and makes the files unreadable and changes on them untraceable. I would prefer a solution that encrypts only the variable values. That way, encrypted files could be committed in repositories and continue to have diff logs that are meaningful (for history, auditing and troobleshooting). For example, you could easily see that Bob changed “mysql\_root\_password” or that Alice removed some credentials. Also, I think that implementing this would have no impact on any of the suggested UIs. Secondly, on the UI: Michael’s interface proposal looks simpler. However, it does not provide information on how multiple passwords can be used (in parallel) for different files/variables.

[Previous page](https://forum.ansible.com/t/secret-variables-data-encryption/13042.md?page=2)

[Next page](https://forum.ansible.com/t/secret-variables-data-encryption/13042.md?page=4)
