pretty printing 'complete' dictionary vars in a file after reading from a list of dicts

Archives Ansible Project
1

I have a large number of YAML files prepared manually which have a fixed structure - list of dictionaries. But within different files, sequence of these keys and their values differ. I need to manage them automatically, so i am writing a kind of converter to add marker blocks. But i am stuck at this point:

example input YAML file

`

#something

keyA:
A1:

  • bla
  • bla
    A2: bla bla
    A3:
    A3.1: bla
    A3.2: bla

keyA:
C1:

  • bla
    C2: bla bla
    C3:
    C3.1: bla

keyB:
B1:

  • bla
  • bla
    B2: bla bla
    B3:
    B3.1: bla
    B3.2: bla
    B3.3: bla
    B4:
  • bla
  • bla

`

playbook to load one of the YAML, and try to write the same variables again with markers appended

`

  • include_vars:
    file: “path to one of the YAML file”
    name: yamlVars

  • template:
    src: updated.yaml.j2
    dest: “path to that same file”

`

the template

`

{% for key, value in yamlVars.iteritems() %}

BEGIN BLOCK {{ key }}

{{ value | to_nice_yaml }}

END BLOCK {{ key }}

{% endfor %}

`

Poblem is, always the contents of each key is printed:

(manually prepared output to hide my variables)

actual output

`

BEGIN BLOCK keyA

A1:

  • bla
  • bla
    A2: bla bla
    A3:
    A3.1: bla
    A3.2: bla

END BLOCK keyA

BEGIN BLOCK keyC

C1:

  • bla
    C2: bla bla
    C3:
    C3.1: bla

END BLOCK keyC

BEGIN BLOCK keyB

B1:

  • bla
  • bla
    B2: bla bla
    B3:
    B3.1: bla
    B3.2: bla
    B3.3: bla
    B4:
  • bla
  • bla

END BLOCK keyB

`

expected output

`

BEGIN BLOCK keyA

keyA:
A1:

  • bla
  • bla
    A2: bla bla
    A3:
    A3.1: bla
    A3.2: bla

END BLOCK keyA

BEGIN BLOCK keyC

keyC:
C1:

  • bla
    C2: bla bla
    C3:
    C3.1: bla

END BLOCK keyC

BEGIN BLOCK keyB

keyB:
B1:

  • bla
  • bla
    B2: bla bla
    B3:
    B3.1: bla
    B3.2: bla
    B3.3: bla
    B4:
  • bla
  • bla

END BLOCK keyB

`

2

I think you want the following:

{% for key, value in yamlVars.iteritems() %}

BEGIN BLOCK {{ key }}

{{ key }}:
{{ value | to_nice_yaml | indent }}

END BLOCK {{ key }}

{% endfor %}

http://jinja.pocoo.org/docs/dev/templates/#indent

3

Hi Matt,

Yes, you are thinking in the right direction. Unfortunately it is still not working as intended. The indentation generated is crazy.

Basically i want idempotency by maintaining the actual indentation of the complete file always.

4

You are going to have to not use vars files then. Or you will need to store the data as strings.

If you don’t want formatting to change at all, you can’t convert from YAML to python dict, to YAML, to jinja2+YAML. You will need to find a way to join them without going through those steps.

5

Hi,

To add markers in all of them, i either have the option of lineinefile with regex or this. Regex for such a case look almost impossible. So i have to make this work.

When i print the complete yamlVars, it works perfectly fine. Can i not append each key-value pair to a temporary dictionary and print that dictionary ?

6

Have you tried this for 2 space indentation

# BEGIN BLOCK {{ key }}
{{ key }}:
  {{ value | to_nice_yaml(indent=2) }}
# END BLOCK {{ key }}

for 4 space you can use this
# BEGIN BLOCK {{ key }}
{{ key }}:
    {{ value | to_nice_yaml(indent=4) }}
# END BLOCK {{ key }}

You will never managed to create this with to_nice_yaml
keyA:
  A1:
    - bla
    - bla

Since the correct 2 space indentation is
keyA:
  A1:
  - bla
  - bla

But if you can live with that the code above should work.

Another thing, dictionary is not sorted so you can get different order of the keyX between subsequent runs.

7

Hi,

More indentation doesn’t work. Maybe i should think of some other approach entirely.
Thanks for your help.

BR,
Ishan

8

I have done similar things. With enough “experimentation”, you can get it right.

Here’s an example from a recent project:

`