Ansible is capable of many things

This is a first post in the series of posts (I hope) about little project I’ve been working on.

Idea of the project is to demonstrate ansible capabilities. In my experience I see many misunderstandings (from my point of view) what ansible-core is capable of and subsequent misuse of ansible-core functionality. This makes ansible content (here I mean yaml content) much more complex to understand, change and write than it suppose to be.

When I’ve started learning ansible I’ve lerning a lot from the community posts and content, this is my way of giving back to community. And I am looking forward to your feedback (positive or negative).

I would like to introduce capable.core212 collection. I’ve released versions 1.0.0 today.
Let me clarify the naming - namespace = “capable” indicates that ansible is capable of many things, name = “core212” indicates that all plugins and modules (all content) in this collection shoud work with all ansible-core versions starting from 2.12. There will be more (capable.core219 is partly done already, but not yet ready to be published).

For the first content in this collection credit goes to @utoddl and his logical plugin (GitHub - utoddl/logical: A filter for Ansible that adds if / elif / else and logical operators to Ansible data. · GitHub). I’ve changed a syntax a bit, but kept majority of the code and logic.

Ever wondered how to write if, then, else conditon in ansible? Used with_first_found to do that (this is bad practice from my point of view), used other complex setups to do the same thing?

Now you can just write if/then/else in simple yaml.
Here are examples:

---
# Simple if/then/else conditional
- name: simple conditional
  vars:
    data:
      if: true
      then: purple
      else: green
    result: "{{ data | logical }}"
    # result == "purple"

# if/then/elseif/else - first matching branch wins
- name: elseif chain
  vars:
    data:
      if: false
      then: first
      elseif:
        - elseif: true
          then: second
        - elseif: true
          then: third
      else: last
    result: "{{ data | logical }}"
    # result == "second"

# Nested conditionals - then/else values can themselves contain conditionals
- name: nested conditionals
  vars:
    data:
      server:
        if: true
        then:
          region:
            if: false
            then: us-east-1
            else: eu-west-1
          size: large
        else:
          region: default
          size: small
    result: "{{ data | logical }}"
    # result == {server: {region: "eu-west-1", size: "large"}}

# Complex then/else values - dicts and lists as results
- name: dict as then value
  vars:
    data:
      if: true
      then:
        name: server1
        port: 8080
        tags:
          - web
          - production
      else:
        name: fallback
        port: 80
    result: "{{ data | logical }}"
    # result == {name: "server1", port: 8080, tags: ["web", "production"]}

- name: list as then/else value
  vars:
    data:
      if: false
      then:
        - item1
        - item2
      else:
        - fallback1
        - fallback2
        - fallback3
    result: "{{ data | logical }}"
    # result == ["fallback1", "fallback2", "fallback3"]

# Missing else - returns None when condition is false and no else provided
- name: no else clause
  vars:
    data:
      if: false
      then: unreachable
    result: "{{ data | logical }}"
    # result == None

# Mixed static and conditional keys - regular keys pass through unchanged
- name: passthrough with nested conditional
  vars:
    data:
      static_key: static_value
      nested:
        if: true
        then: resolved
    result: "{{ data | logical }}"
    # result == {static_key: "static_value", nested: "resolved"}

# String booleans from Jinja2 templating
- name: string booleans
  vars:
    my_flag: true
    data:
      if: "{{ my_flag }}"
      then: resolved true
      else: resolved false
    result: "{{ data | logical }}"
    # result == "resolved true"

# Logical operators standalone
- name: and operator - true if all are true
  vars:
    data:
      and:
        - true
        - true
        - true
    result: "{{ data | logical }}"
    # result == true

- name: or operator - true if any is true
  vars:
    data:
      or:
        - false
        - true
        - false
    result: "{{ data | logical }}"
    # result == true

- name: xor operator - true if exactly one is true
  vars:
    xor_true:
      xor:
        - false
        - true
        - false
    xor_false_none:
      xor:
        - false
        - false
        - false
    xor_false_many:
      xor:
        - true
        - true
        - false
    # xor_true | logical == true  (exactly one)
    # xor_false_none | logical == false  (zero true)
    # xor_false_many | logical == false  (more than one true)

- name: not operator - flips the boolean value
  vars:
    data:
      not: false
    result: "{{ data | logical }}"
    # result == true

# Logical operators as conditions in if
- name: and as condition
  vars:
    data:
      if:
        and:
          - true
          - true
      then: and passed
      else: and failed
    result: "{{ data | logical }}"
    # result == "and passed"

- name: or as condition
  vars:
    data:
      if:
        or:
          - false
          - true
      then: or passed
      else: or failed
    result: "{{ data | logical }}"
    # result == "or passed"

- name: not as condition
  vars:
    data:
      if:
        not: false
      then: not passed
      else: not failed
    result: "{{ data | logical }}"
    # result == "not passed"

# Nested operators in condition
- name: nested operators
  vars:
    data:
      if:
        and:
          - not: false
          - or:
              - false
              - true
          - xor:
              - true
              - false
      then: all passed
      else: something failed
    result: "{{ data | logical }}"
    # result == "all passed"

What do you want me to release next?

  • validate_vars plugin - add vadation rules of any complexity to your ansible roles
  • how to write and use python functions in ansible-core 2.19
  • include_fact plugin - same as set_fact, but uses files similar to include_vars
  • set_vars plugin - same as include_vars, but uses parameters similar to set_fact
  • how to create and publish modern documentation website for ansible collection
0 voters

Am I missing something with the need for a plugin?

jinja has logical checking built in.


- name: Jinja logic
  vars:
    my_var: |-
      {%  if var_to_test == 'expectedValue' -%}
        value for true
      {% else -%}
        value for false
      {% endif %}

2 Likes

That is true.

And jinja as text templating engine will return text. That text will have to be converted (implicitly and differently by different ansible-core versions) by ansible from json to whatever structure your need (list, dict, etc)

Here native ansible values are returned. So no one has to think if ‘[1,2,3]’ returned by jinja is a list or plain string (due to missed or extra space).

Promotions. You and this stripped down version of the logical filter are missing promotions with optional merging (dicts) and joining (lists), without which you can (almost) do the same thing with Jinja.

1 Like

They are missing in my version of logical filter. I’ve changed interface a bit.

Instead of list, logical filter takes following dict:

logical_condition:
   if: 
   then:
   [elseif:]
   else:

elseif key is optional.

I found it to be more precise representation of if/then/else construction.

Oh, right: dict (i.e. order doesn’t matter) in the capable.core212.logical filter vs. list items (order does matter) in the utoddl.logical.logical filter. I glided right over that difference and didn’t even notice on first reading. Oops.

Also, naming things is hard™. I had a working stand alone logical filter before collections were a thing, then leaped - perhaps prematurely - onto the collection bandwagon as soon as that became practical. I ended up with a collection with a stupid name containing only the one filter of the same name. Similarly, I suspect “core212” as a component in your FQCN is going to be of dwindling interest rather soon. But I guess that’s partly the point.

I will say this: Your code is way cleaner than mine, and that’s only partly because mine includes data promotions. Mine is also about ⅓ debugging print statements. There’s a reason my code looks like it was written by somebody who learned BASIC in the late '70s and hadn’t really grokked Python when he started writing Ansible filters. But since it works and life is short enough already, I haven’t prioritized cleaning it up.

To @jon-nfc’s point, which paraphrased is, “Why embed logic in data outside of the Jinja provided methods?” @kks already mentioned Jinja is geared toward text, although Ansible is much smarter about interpreting Jinja output as data than it was at the time utoddl.logical.logical (ne logical) was written. Likewise the number and quality of Ansible filters etc has vastly improved since then. Even so, for me it comes down to, “Which expression of the data and logic would I rather maintain?”

If anyone finds logical (any variant) abhorrent, then by all means don’t use it. But for those intrigued by the idea of data altering itself, then I invite you to take a look at the polymac filter!

1 Like

As it is with jinja

what is returned is what you place. Although as you point out the “missed” or “extra” space is a problem. To address this formatting your jinja correctly is required. As for the stated confusion of a list or string, again thia comes down to correctly defining jinja.

um, no it wasnt. paraphrased us why do or would i need another module for logic when i can achieve logic via jinja

To release oneself from the shackles of jinja.

Seriously if jinja works for you, please use it. For me it does not for various reasons. Not the last one is that jinja is a “language” not data, I cannot easily do manipulations with it, it is fragile.

My ansible experience tells me to keep as much distance as possible from jinja.

1 Like

I certainly felt that way when I wrote utoddl.logical.logical, but I’ve gotten a lot better at Jinja since then, and Jinja is much better integrated into Ansible/YAML now, too. I wouldn’t want to overstate that case.

Particularly, here’s a heart-felt shout-out :heart: to the core team members involved in making Ansible so much more pleasant to use over the last decade. Seriously: If I’d had then the Ansible we have now, I probably wouldn’t have written utoddl.logical.logical in the first place.

But I still use it in some places. Here’s an example from a project group_vars file:

_slprj_packages_install_all:
  - jq
  - python3-setuptools
  - if:  # rhel8 ships these in the codeready-builder repo
      - "{{ ansible_facts.distribution_major_version >= '8' }}"
      - name:
          - python3-libselinux
          - python3-virtualenv
        enablerepo: codeready-builder-for-rhel-{{ ansible_facts.distribution_major_version }}-x86_64-rpms
      - name:
          - libpng15
          - rpmdevtools
          - python39
          - python3.12
        enablerepo: rhel-{{ ansible_facts.distribution_major_version }}-for-x86_64-appstream-rpms
      - name:
          - python3.11
          - python3.11-setuptools-wheel
          - python3.11-pip-wheel
          - python3.11-wheel-wheel
          - python3.11-libs
        state: absent

slprj_packages_install_all: "{{ _slprj_packages_install_all | utoddl.logical.logical }}"

_slprj_pip_install_packages:
  - if:   # These are for RHEL 8 and above
      - "{{ ansible_facts.distribution_major_version >= '8' }}"
      - virtualenv: /opt/venv-erds-python39
        virtualenv_python: python3.9
        name: &piplist
          - ansible-lint
          - ansible-tower-cli
          - awxkit
          - boto
          - dnspython
          - infoblox-client
          - jmespath
          - jq
          - kubernetes
          - mypy-boto3
          - netaddr
          - netapp-lib
          - ntlm-auth
          - psutil
          - psycopg2-binary
          - pyvmomi
          - pywinrm
          - selinux
          - setuptools_rust
          - virtualenv
          - pip
      - virtualenv: /opt/venv-erds-python312
        virtualenv_python: python3.12
        name: *piplist

# If you need to remove pips that are already installed, you'd do this:
#   - name:
#       - cx-oracle
#     state: absent
#     virtualenv: /opt/venv-erds-python39

slprj_pip_install_packages: "{{ _slprj_pip_install_packages | utoddl.logical.logical }}"
5 Likes

This… Which leads into below.

This paragraph pointed out an important part of either end of the same spectrum. For me Jinja logic is pythonic with “extra pain points to formatting”. As someone with a programming background, jinja usage is natural especially after learning the jinja nuances. Though my own ignorance I didn’t identify this as an issue for anyone else. On the flip side, you (and quite possibly a majority of Ansible users) are on the other side of the spectrum to me, Hence face challenges I or others like me either cant identify (due to not experiencing, no exposure etc) or have no understanding.

From my perspective: I have never found anything “logic wise” that cant be achieved through jinja. Additionally I do use jinja for data manipulation, examples:

These examples in no way provide that the plugin in question is not required. On the contrary, I’m of the opinion it is. Why? I look at the plugin and don’t understand why it’s needed when there is a perfectly usable method already. Again on the flip side, someone else would look at the Jinja equivalent and one, quite possibly be confused and two, not understand how to get/keep it working. The former is not the target audience for the plugin, the latter is.

@utoddl

I’ve just realised that you are the “creator” of the subject plugin. Kudos. With what I’ve discovered throughout this discussion I am of the opinion that a very suitable reason for your plugin plus a good selling point is the following:

  • Provided a means to perform logical functions that is: simpler, native and without the use of Jinja (or even replaces Jinja).

This covers both ends of the spectrum. so now two different users from any end of the spectrum whom discover your work, can determine, “yes” I need or “maybe” I don’t.

From your comments It appears as if you may have started as someone whom this plugin was designed–to later become someone whom may no longer require it.

1 Like

Promotions? the what now? you’ll have to define what you mean by this term

To merge, create, filter lists and dicts in Jinja ensure the jinja output must return a valid json object of what ever you are trying to return. Key to success for this, however and due to scopes in jinja, you must use namespaces, i.e. {% set ns = namespace(added_teams = []) %} which would become var ns.added_teams within the jinja. see → ansible_collection_centurion/playbooks/teams.yaml at 3341ac1cc4dd8dfedb8b432c8643644c30ab2576 · nofusscomputing/ansible_collection_centurion · GitHub as a method of how I achieved.

Heh heh. Okay. This isn’t structured well, but I have to cover a couple of background points. They’re both about context.

Thing the 1st: We’re talking about two fundamentally different ways to implement data logic: (1) strings which Jinja crunches to produce grokable YAML (a subset of which is JSON), and (2) YAML data containing embedded conventions which, when processed, produce a subset of that data.

Just to blur that distinction as much as possible, the processing of #2 happens to be implemented as a Jinja filter. As a side effect of that implementation, #2 can also contain Jinja expressions. But strictly speaking it’s a YAML thing, not a Jinja thing.

Thing the 2nd: It’s all about context. We like to pretend that our data definitions (and code) are strictly block structured, that inner blocks can’t “pollute” outer blocks. Earlier Jinja was particularly onerous about that, so people (myself included) would use side effects (.pop(), .append()) to get around those strictures. Jinja added namespaces to provide a “legal” means to achieve the same thing, only by using them you’re basically saying, “I know what I’m doing and it’s on purpose so don’t send the scope police after me.”

The idea isn’t restricted to Jinja either. Some way to achieve “scope violations on purpose” eventually gets implemented wherever scopes are implemented.

Promotions (see links in prior posts) provide a way to take the logical result and inject it into its surrounding context, pushing it up a level one way or another. I’ve only studied this for utoddl.logical.logical; I haven’t looked at such issues for capable.core212.logical.

As an example, consider this data below.


Note: this code is from before “lazy evaluation” was the norm. Back then, once
soup: "{{ ['too hot', 'too cold', 'just right'] | random() }}"
was evaluated, its value was static. In fact, the value of any variable which was set to that string would also have the same value as soup. That is, the expression
weather: "{{ ['too hot', 'too cold', 'just right'] | random() }}"
would give the same static value as soup, and the random() filter would not be evaluated more than once. You could get a different random-but-static value for weather by changing the Jinja string, even by simply adding an extra space, like so:
weather: "{{ ['too hot', 'too cold', 'just right' ] | random() }}"
With today’s lazy evaluation, soup takes a new random value from the list each time it is evaluated which kind of breaks the demo as far as the Goldilocks story goes, but the logical issues are unchanged


  vars:
    soup:  "{{ ['too hot', 'too cold', 'just right'] | random() }}"
    data:
      soup:
        is: "{{ soup }}"
        owner:
        - if: ["{{ soup == 'too hot' }}","Papa Bear"]
        - elif: ["{{ soup == 'too cold' }}","Mama Bear"]
        - else: "Baby Bear"

Because if, elif, and else have to be list items (for utoddl.logical.logical; not capable.core212.logical), they naturally result in a list item. That’s fine if what you want is a list item. In this case, though, not so great. In fact,
"{{ data | utoddl.logical.logical }}"
produces this:

  soup:
    is: too cold
    owner:
    - Mama Bear

After applying the logical filter, the "owner will have become a single element list. We’d like to promote the “owner” value up a level, thus:

  soup:
    is: too cold
    owner: Mama Bear

The << operator will achieve that.

      soup:
        is: "{{ soup }}"
        <<owner:
        - if: ["{{ soup == 'too hot' }}","Papa Bear"]
        - elif: ["{{ soup == 'too cold' }}","Mama Bear"]
        - else: "Baby Bear"

The rules for promoting dicts into dicts with merging or replacement, or lists into lists with appending or uniquification by using the <<, <<|, and <<- operators are tricky and too long to go into here. Again, see prior links.

But that’s promotion in a nutshell.
(Hey, you asked, and I’m lousy at short answers!)

1 Like

So if I’m understanding correctly, provides for being able to declare data types list and dict whn doing logic

Not so much a declaration; that sounds way less hacky than it feels. Maybe you can think of it that way. But the data types are set regardless of or in the absence of promotions. To me it’s more about how the resulting data gets grafted onto or into the next higher level data structures.

I say, look at the docs, then be glad you don’t need it!