Question about error handling in loops with lookups

Get Help
1

I have the following code:

set_fact:
  accounts: "{{ accounts + query('accountslookup', resource=item) }}"
loop: "{{ resourcelist }}"

This generates a list of accounts in multiple resources.

This works fine, until there’s a resource that doet not exist.

In Ansible 2.18 that would result in a useful error:

fatal: [localhost]: FAILED! => {"msg": "An unhandled exception occurred while running the lookup plugin 'accountslookup'. Error was: [usefulerrormessage from lookup plugin] [...].

In Ansible 2.19 this has been replaced with:

[ERROR]: Task failed: Finalization of task args for 'ansible.builtin.set_fact' failed: Error while resolving value for 'accounts': Error rendering template: can only concatenate list (not "CapturedExceptionMarker") to list

and the original error is not available

Two questions:

  • Are there better ways to loop over a list and generate a joined array of items?
  • Is there a setting somewhere that will give me the original error instead of hiding it with a ‘CapturedExceptionMarker’ message?
2

I do not know what accountslookup does and what it returns in case of error. Error handling of course depends on that.
But this seems to me a bit more robust approach:

set_fact:
  accounts: "{{ accounts + new_accounts }}"
loop: "{{ resourcelist }}"
vars:
   new_accounts: "{{ query('accountslookup', resource=item) | default([]) }}"
3

The lookup plugin raises an exception (raise ValueError('resource not found'))

4

I suspect accountslookup is custom lookup plugin.
I see two options here:

First - do not raise an exception. Not sure if it is mentioned in the development guide, but this seems to me like a bad idea to raise an expection inside lookup plugin - ansible is not great tool to handle exceptions. Better is just return empty list, or another form of absent value.

Second options is much more ugly. If my code from above does not help, you have to wrap lookup plugin call in block with rescue section. And this block should be exectuted in the loop. As you might know ansible documentation says that loop is not supported:

“All tasks in a block inherit directives applied at the block level. Most of what you can apply to a single task (with the exception of loops) can be applied at the block level, so blocks make it much easier to set data or directives common to the tasks.”

Despite of that it is actually possible to run block statements inside a loop - you just have to have block (and rescue) section inside a task file and include that file with include_tasks module - that would be a single task, that supports loop directive. Inside included task file (and inside block section) one can use item special variable (or rename it for clarity).

So you run loop each iteration, that includes block (in separate task file). Block task - set_fact, if lookup fails, use rescue section, so playbook does not fail. Section has to be present, but do nothing, just “swallow” error.

5

|default() after the query() is too late, as accountslookup has already failed at that point.

It may be possible to get accountslookup to be more benign if you default item to an empty dict or list or string, or maybe even a harmless (?) dummy account, something like:

vars:
   new_accounts: "{{ query('accountslookup', resource=(item | default({}))) }}"

Or not. Might be worth trying, though.

6

I don’t want it to fail silently, I do want to stop with an error. But since 2.19 the actual error is hidden.

7

In that case, I can think of three different approaches. They aren’t necessarily mutually exclusive.

  • modify the lookup so it provides something you can work with
  • vet the looped-over resourcelist prior to the query / set_fact
  • use a sentinel default value that prevents the query from crashing but which you can test for in the resulting ansible_facts.accounts after the query / set_fact
8

Ok, got it
In that case something like this:

block: Set accounts
set_fact:
  accounts: "{{ accounts + new_accounts }}"
loop: "{{ resourcelist }}"

rescue:
debug:
  msg: "Failure, this is what query returned: {{ new_accounts | string }}"
vars:
   new_accounts: "{{ query('accountslookup', resource=item) | default([]) }}"

Again it depends what accountslookup returns in case of error.
vars in this case - block vars.

I would like to highlight that lookups should not fail at all in my opinion, they should always return value.
Whole situation looks like swimming against the current.