short-circuit eval in templated string works only sometimes?

Archives Ansible Project
1

Hello, why does short circuit only works for the first two cases:

- hosts: localhost
  become: false
  vars:
    ivar: '{{hostvars[inventory_hostname].dne}}'
  tasks:
    - debug:
        msg: "works: {{'foo' or dne}}"
    - debug:
        msg: "works: {{'bar' or hostvars[inventory_hostname].dne}}"
    - debug:
        msg: "fails: {{'baz' or ivar}}

The third debug will generate an error:

    The task includes an option with an undefined variable.
    The error was: {{hostvars[inventory_hostname].dne}}:
    'ansible.vars.hostvars.HostVarsVars object' has no attribute 'dne'

Why is it even looking up in hostvars at all? The non-empty
string should evaluate True, so why is it reading the ORed
condition? The same thing happens with "if true else" and
ternary(true, ...) filter. It seems that it's properly
short-circuiting in the first two cases, because the variable
"dne" does not exist, and there is no error. Even more
confusing, the expansion of host vars is the exact same in cases
2 and 3, it's only going through an indirect variable in case 3.

This seems to happen regardless of the source of variables ie
extra vars, vars_files, etc.

Is this a bug, or what am I missing?

2

Why don't you start debugging the variable?

  - debug:
      var: dne

See "Minimal working example"
https://en.wikipedia.org/wiki/Minimal_working_example

3

> Hello, why does short circuit only works for the first two cases:
>
> - hosts: localhost
> become: false
> vars:
> ivar: '{{hostvars[inventory_hostname].dne}}'
> tasks:
> - debug:
> msg: "works: {{'foo' or dne}}"
> - debug:
> msg: "works: {{'bar' or hostvars[inventory_hostname].dne}}"
> - debug:
> msg: "fails: {{'baz' or ivar}}
>
> The third debug will generate an error:
>
> The task includes an option with an undefined variable.
> The error was: {{hostvars[inventory_hostname].dne}}:
> 'ansible.vars.hostvars.HostVarsVars object' has no attribute 'dne'
>
> Why is it even looking up in hostvars at all? The non-empty
> string should evaluate True, so why is it reading the ORed
> condition? The same thing happens with "if true else" and
> ternary(true, ...) filter. It seems that it's properly
> short-circuiting in the first two cases, because the variable
> "dne" does not exist, and there is no error. Even more
> confusing, the expansion of host vars is the exact same in cases
> 2 and 3, it's only going through an indirect variable in case 3.
>
> This seems to happen regardless of the source of variables ie
> extra vars, vars_files, etc.
>
> Is this a bug, or what am I missing?

Why don't you start debugging the variable?

  - debug:
      var: dne

See "Minimal working example"
https://en.wikipedia.org/wiki/Minimal_working_example

If the variable *dne* is not defined the playbook

> cat playbook.yml
- hosts: localhost
   vars:
     ivar: "{{ hostvars[inventory_hostname].dne }}"
   tasks:
     - debug:
         var: ivar
     - debug:
         msg: "{{ 'foo' or dne }}"
     - debug:
         msg: "{{ 'foo' or ivar }}"

gives

> ansible-playbook playbook.yml
...
TASK [debug]

4

third debug task fails because the expansion of the string
"{{ ... }}" fails before the *or* expression could be evaluated.

Sure, but why is it expanded at all? You're right that we can
make my already-minimal test when variable Does Not Exist
(dne) even shorter without needing to use hostvars:

- hosts: localhost
  become: false
  vars:
    ivar: '{{dne}}'
  tasks:
    - debug:
        msg: "{{'foo' or ivar}}"

You'll need a default value if you want to use this kind of
"lazy evaluation" constructs ...

I had thought ansible variables were lazily evaluated already.
In docs/docsite/rst/reference_appendices/glossary.rst it says:

    Lazy Evaluation
    In general, Ansible evaluates any variables in playbook
    content at the last possible second, which means
    that if you define a data structure that data structure itself
    can define variable values within it, and everything "just
    works" as you would expect. This also means variable
    strings can include other variables inside of those strings.

This seems to imply that things which aren't ever referenced,
will never be expanded. There is even an open issue with a
feature idea to force evaluation to be non-lazy:
https://github.com/ansible/ansible/issues/10374

I cannot use a default because this whole thing is relying
on whether it's defined or not later to take different actions.
the fuller example is:

- name: determine_subnet
  when: >
    project == project_bake
    or hostvars[target].ipnet is defined
  set_fact:
    subnet: '{{
      (project == project_bake) | ternary(
        subnet_bake, determined_subnet
    )}}'

the problem here is that determined_subnet is defined like this:

    determined_subnet: '{{subnets[ip_cidrnet]}}'

and the chain for that is:

    ip_host: '{{hostvars[ip_referent].ipnet}}'
    ip_referent: '{{host | d(target | d(inventory_hostname))}}'
    ip_network: "{{ip_host | ipaddr('network')}}"
    ip_cidrnet: '{{ip_network}}/{{ip_prefix}}'

When ipnet (an inventory variable) is not defined,
determined_subnet is still being expanded, even though
project == project_bake, and should therefore never need
to evaluate the second arg to ternary(). For this case, ipnet
is not set.

I've made it work by splitting it into two set_facts with
cascading 'when' clauses, but I don't understand why it's
happening in the first place. The evaluation does not seem
to be lazy, or it would never need to evaluate an unused arg
right? And it's the same behavior using ternary, if/else, or
a disjunction ("or").

5

What value of *subnet* do you want to set when '(project !=
project_bake) and (determined_subnet is undefined)' ?

6

When the corresponding alternatives shall always be defined the
defaults can't hurt, e.g.

- set_fact:
     subnet: '{{ (project == project_bake)|
                  ternary(subnet_bake|default(None),
                          determined_subnet|default(None) }}'

In this case test sanity first

- fail:
     msg: Variable undefined
   when: ((project == project_bake) and (subnet_bake is undefined)) or
         ((project != project_bake) and (determined_subnet is
         undefined))

7

determined_subnet is always defined, except for project_bake.
project_bake nodes don't have ipnet set (or any inventory vars),
so the subnet url (for cloud api calls) can't be determined. So
it gets hardcoded for that project. Hence, the need for the
condition.

8

Unfortunately that doesn't work, I tried it before. Because it's
so indirect, it needs a default at the level that gets referenced,
not the outer level of the indirection. It gets:

localhost failed | msg: The task includes an option with an
undefined variable. The error was: {{subnets[ip_cidrnet]}}:
{{ip_network}}/{{ip_prefix}}: {{ip_host | ipaddr('network')}}:
{{hostvars[ip_referent].ipnet}}:
'ansible.vars.hostvars.HostVarsVars object' has no attribute
'ipnet'

If only it would not try to dereference the thing, because it's
never actually consulted for a value, as it's the OR with a
true condition. That's the part I still don't get, why it's
happening at all.

Anyways, I have worked around it at this point by splitting
into two separate conditions with "when". I just wanted
to understand why. I still don't understand why, but that
happens sometimes... thanks.

9

This is effectively a side-effect of functionality in Ansible's
templating engine that allows nesting variables, like `ivar:
'{{hostvars[inventory_hostname].dne}}'` in your example. The second
operand of `or` is actually resolved somewhere within Jinja templating
machinery in all three cases. The difference is with the third one
where Ansible's extended templating functionality gets `ivar` and
tries to template it *because it is a template* (=this is what allows
nesting variables) which causes the failure.

There are issues reported asking to change this behavior, see
https://github.com/ansible/ansible/issues/58835 and
https://github.com/ansible/ansible/issues/56017 (or an interesting
example that was filed recently in
https://github.com/ansible/ansible/issues/74594).

The problem is that while changing this might result in expected
behavior in this case, it will break other scenarios. However the
above issues are open so feel free to give your input there.

Hopefully this clarifies the matter a bit.

Thanks,
Martin

10

This is effectively a side-effect of functionality in Ansible's
templating engine that allows nesting variables ...

FWIW, one more of this kind not mentioned yet, I think

    - debug:
        var: my_dict.a
      vars:
        my_dict:
          b: "{{ undefined_var }}"

gives

    my_dict.a: VARIABLE IS NOT DEFINED!