Parsing and vaulting yaml the Ansible way with ruamel.yaml

I tend to favor vaulted scalar values in vars files over entire vaulted files, partly because I can grep for variable names successfully. (I grep a lot!) But creating and managing vaulted scalars is a bit clunky.

To make it less clunky, I wrote a simple filter in Python that can be invoked from within an editor. Select the lines of yaml containing your keys and values, pass that block of yaml through the filter’s stdin (most text editors have a way to do that), and it will use ruamel.yaml to parse the yaml, process it, and return the result through its stdout. It will vault for you any unvaulted scalars it finds, and unvault any named vaulted data it finds, all while preserving your indentation. You can give it the optional parameter of the vault identity you’d like it to use for vaulting; otherwise it will use whatever your ANSIBLE_VAULT_IDENTITY environment variable is set to. Underneath of course it’s calling ansible-vault to do the actual vault work.

“Sounds great! What’s the problem?” Yeah, well, it isn’t particularly kind to comments, for one thing. It would be really cool to make it “vault→unvault round-trip idempotent”, which it mostly is if comments aren’t involved.

Another thing, which will be obvious if you look at the code, particularly around lines 252 and following, is that I totally failed at making ruamel.yaml vault-aware. My Python skills are of the cargo-culture copy-n-paste variety. And my staring at Ansible’s own code and the ruamel.yaml docs has yielded no enlightenment. The working code as it is feels really hacky.

Even so, I think this is a very useful tool, it’s dead simple to use, and it basically works. So here’s my “ask”, which I realize is perhaps bigger than I could expect anyone to volunteer for, in easy to “don’t even think about it” order:

  • I’d like someone familiar with Python to give this a once-over (it’s right at 300 lines) and point out the most embarrassing non-Pythonic warts.
  • I’d like someone who understands ruamel.yaml well enough to make it parse and emit !vault values like it ought to.
  • If possible, I’d like to know how to use the components of the installed Ansible suite to do this as more of an extension of Ansible rather than the awkward imitation work-alike that it currently is. (It already requires having ansible-vault installed. Requiring ansible-lint or whatever else is fair.)

Finally, after any or all of this is done, I’d like it to still be as dead simple to use as it currently is.

1 Like