Variable considered undefined if it contains undefined values

If a variable exist, but contains undefined value. Its considered undefined.
In the past this would create a error.

  • Is this a bug?
  • and if it works as intended is there a way to get the old behavior?

problem

For example:

test.j2:

{% if a is defined %}
{{ a[0] }}
{% endif %}
a: [5,{{b}}]

If you run this with template module:

  • In the past this gave a error AnsibleUndefinedVariable for b
  • Now, it results in a empty file. because it considers variable a undefined.

Now if someone makes one mistake when changing a variable.
whole blocks of configuration are dropped, without a error.
We had a example were most of our firewall rules got silently dropped.

versions used

$> ansible --version
ansible [core 2.15.3]
  config file = /etc/ansible/ansible.cfg
  ...
  executable location = /usr/bin/ansible
  python version = 3.11.5 (main, Sep 22 2023, 15:34:29) [GCC 8.5.0 20210514 (Red Hat 8.5.0-20)] (/usr/bin/python3.11)
  jinja version = 3.1.2
  libyaml = True

Have not yet found out which version of ansible still gave the error.
But i have confirmed it at least gave error in past. by trying it in centos7 container

I have done a git bisect to find which commits introduced this change.

cbe42bff7f16082521bf4e4dfd2c3de8f4e8cee4 is the first bad commit
commit cbe42bff7f16082521bf4e4dfd2c3de8f4e8cee4
Author: Martin Krizek <martin.krizek@gmail.com>
Date:   Thu Apr 14 19:32:00 2022 +0200

    Allow for lazy evaluation of Jinja2 expressions (#56116)

 .../56017-allow-lazy-eval-on-jinja2-expr.yml       |  2 ++
 .../rst/porting_guides/porting_guide_core_2.14.rst | 14 ++++++++++++-
 lib/ansible/template/vars.py                       | 10 ++++++++-
 test/integration/targets/template/lazy_eval.yml    | 24 ++++++++++++++++++++++
 test/integration/targets/template/runme.sh         |  2 ++
 test/units/playbook/test_conditional.py            | 13 ------------
 6 files changed, 50 insertions(+), 15 deletions(-)
 create mode 100644 changelogs/fragments/56017-allow-lazy-eval-on-jinja2-expr.yml
 create mode 100644 test/integration/targets/template/lazy_eval.yml
bisect found first bad commit

So change might be between ansible-core 2.12 and 2.14

Does this answer the question?

in ansible-core 2.14 an expression {{ defined_variable or undefined_variable }} does not fail on undefined_variable if the first part of or is evaluated to True as it is not needed to evaluate the second part. One particular case of a change in behavior to note is the task below which uses the undefined test. Prior to version 2.14 this would result in a fatal error trying to access the undefined value in the dictionary

2 Likes

It isn’t easy to throw an informative exception in a jinja template. The only thing I’ve come up with so far is this:

{% if a | type_debug == 'AnsibleUndefined' %}
{%     set a = 1 / 0 %}
{% else %}
{{     a[0] }}
{% endif %}
2 Likes

@chris I understand the use case for short-circuit evaluation for or and and statement. 56116

But I don’t understand why ansible mark a variable as undefined when. only a small part of it`s content undefined.

@utoddl cool type_debug did not know about that one.

But as far as i understand does a | type_debug == 'AnsibleUndefined the same as is defined

what seems to work, bit look like ugly work around:

{% if a is defined and a | json_query('@') }
{{     a[0] }}
{% endif %}

because the filter force evaluation of the entire variable. and triggers a error if it finds a nested undefined

1 Like

Argh! You’re right. That’s no help at all. Hmm.

@Rens_Sikma

Personally, I would have expected the current behavior, and the old behavior would have surprised me, and I’ve been using Ansible since 2.8.

Perhaps it might be a better idea to go over what you’re actually doing, and maybe we can help with structuring your Ansible playbook/role? Good practices with Ansible is to try and provide default() values for any var that might not always be “defined” and then determine what to do when those vars are defined/undefined/default(). A “partially defined” var should be an undefined var, like what you’re seeing now in your template, the jinja2 condition evaluates to false, and nothing is written (rather than erroring because why should “variable is defined” ever return an error?)

1 Like

I appreciate the problem you’ve stumbled into, and the “solution”.

However, if you’re going to define variables in terms of other variables, I think you’d do well to express reasonable defaults wherever that happens. For example (and this stretches the notional of “reasonable”):

# rather than
a: [5, '{{ b }}']

# do this instead
a: [5, '{{ b | default("Noooooo!") }}']
2 Likes

@Denney-tech that might be a good plan, I am still considering to create a gitlab issue/bug-report.
But if there are nice solutions for the problems that i have. that might not be needed.

example

A example where we noticed this change was in our role that mange’s iptables
that manages iptables.

This roles uses variables like:

  • iptables_common_rules_inbound
  • iptables_application_rules_inbound
  • iptables_common_rules_outbound
  • iptables_application_rules_outbound

And this implements by create iptables configuration file

{% iptables_common_rules_inbound is defined  %}
{% for rule in iptables_common_rules_inbound %}                                                                                                                                                                  
# {{ rule.description }}       
{{ rule.iptables_rule }}       
{% endfor %}
{% endif %}

But i think most problems are resolved if this becomes:

{% for rule in iptables_common_rules_inbound %}                                                                                                                                                                  
# {{ rule.description }}       
{{ rule.iptables_rule }}       
{% endfor %}

That works if has a good default

Maybe you could use the mandatory filter ansible.builtin.mandatory filter – make a variable’s existance mandatory — Ansible Community Documentation

{% for rule in (iptables_common_rules_inbound|mandatory) %}                                                                                                                                                                # {{ rule.description }}
{{ rule.iptables_rule }}
{% endfor %}
1 Like