first_found lookup not searching the expected paths

Get Help
1

The roles in this long-existing repository often use a concept like…

- name: COPY | copy 20auto-upgrades
  ansible.builtin.copy:
    src: "{{ _item }}" 
    dest: /etc/apt/apt.conf.d/20auto-upgrades
    backup: no
    owner: root
    group: root
    mode: "0644"
  vars:
    _files:
    - "{{ autoupdate_20auto_upgrades | default([]) }}"
    - "{{ topdir }}/files/autoupdate/{{ inventory_hostname_short }}/20auto-upgrades"
    - "{{ topdir }}/files/autoupdate/{{ ansible_hostname }}/20auto-upgrades"
    - "{{ topdir }}/files/autoupdate/{{ host_group | default(False) }}/20auto-upgrades"
    - "20auto-upgrades.{{ ansible_distribution_release }}"
    _item: "{{ lookup('ansible.builtin.first_found', _files) }}"
  when:
    - ansible_distribution == 'Debian'
    - ansible_distribution_major_version is version_compare('9', '>=')
    - _item is ansible.builtin.truthy
  tags: autoupdate

to express the idea of having a configuration file default to roles/ROLENAME/files/FILENAME but overridable by host group, host, or configuration variable.

(They actually use lookup('ansible.builtin.first_found', _files, skip=True, errors='ignore'), but I removed the kwargs for debugging this.)

I was just adding a .trixie file and wanted to test this, and colour my surprise at the task just being skipped. Adding -vvvvv I saw that:

TASK [autoupdate : COPY | copy 20auto-upgrades] *****************************************************************
task path: /BASEDIR/roles/autoupdate/tasks/main.yml:154
looking for "20auto-upgrades.trixie" at "/BASEDIR/roles/autoupdate/None/20auto-upgrades.trixie"
looking for "20auto-upgrades.trixie" at "/BASEDIR/roles/autoupdate/20auto-upgrades.trixie"
looking for "20auto-upgrades.trixie" at "/BASEDIR/roles/autoupdate/tasks/None/20auto-upgrades.trixie"
looking for "20auto-upgrades.trixie" at "/BASEDIR/roles/autoupdate/tasks/20auto-upgrades.trixie"
looking for "20auto-upgrades.trixie" at "/BASEDIR/playbooks/hosts/None/20auto-upgrades.trixie"
looking for "20auto-upgrades.trixie" at "/BASEDIR/playbooks/hosts/20auto-upgrades.trixie"
looking for "20auto-upgrades.trixie" at "/BASEDIR/playbooks/hosts/None/20auto-upgrades.trixie"
looking for "20auto-upgrades.trixie" at "/BASEDIR/playbooks/hosts/20auto-upgrades.trixie"
[ERROR]: Task failed: The lookup plugin 'ansible.builtin.first_found' failed: No file was found when using first_found.

However, to my reading, this code is equivalent enough to the examples on ansible.builtin.first_found lookup – return first file found from list — Ansible Community Documentation which also refers to Search paths in Ansible — Ansible Community Documentation which indicates that it should indeed search roles/NAME/files/NAME.

But, apparently, something more than that is wrong. Not only failed it to find…

-rw-r--r-- 1 1015 1016 118 Nov 25 23:33 roles/autoupdate/files/20auto-upgrades.trixie

… but the other names and paths aren’t searched either.

What am I missing from the documentation that explains why this does not look up as intended?

2

While the first link to the examples of ansible.builtin.first_found seems to indicate that the plugin should search in files/, the second link (Search paths in Ansible — Ansible Community Documentation) does not say that find_first will look in files/. It generally explains the file search order, and mentions files/ only next to “in its appropriate subdirectory”. (This applies to the action plugin ansible.builtin.copy for example, which looks in files/.)

Which version of ansible-core are you using? Maybe the behavior for find_first changed over time (and the find_first docs were not updated), or maybe the find_first docs were always wrong…

3

Hm, this is… a bit annoying. Other things do find their files under roles/ROLE/files/NAME, now I’m wondering which don’t.

This is Ansibe from Debian trixie (2.19.4), used to be bookworm (2.14.18).

I’m not sure that this code was always written like that. I’m also seeing uses of this pattern:

- name: CONFIGURE | ntp config
  template: src="{{ item }}" dest=/etc/ntp.conf backup=yes
  with_first_found:
    - "{{ common_ntp_conf }}"
    - "{{ topdir }}/files/common/{{ inventory_hostname_short }}/ntp.conf.j2"
    - "{{ topdir }}/files/common/{{ ansible_hostname }}/ntp.conf.j2"
    - "{{ topdir }}/files/common/{{ host_group | default(False) }}/ntp.conf.j2"
    - "ntp.conf.j2.{{ ansible_os_family_release }}"
    - ntp.conf.j2
  notify:
    - restart ntp
  when:
    - ansible_distribution == 'Debian'
    - ansible_distribution_major_version is version_compare('10', '<=')
  tags:
  - ntp
  - common

I’m not sure which is older (I can probably dig it out of git if needed) or which is better. The one from the original post has the added benefit of having the selected filename in a variable, so that extra actions can be done based on that, including skipping if no file was found (whether that is desired or not is a per-task decision). Perhaps the two patterns did emerge because they needed this for some tasks.

Can you recommend way(s) to fix this? Such as, adding the pathe to the lookup somehow (can I put them into a more global variable, ideally one that even selects the current role automatically, but one in roles/ROLE/defaults/main.yml would also work), or refactoring it to use a better pattern for this kind of search?

This should ideally still work with Ansible 2.14 for a while, until the last target nodes that have older Python3 versions are gone.

4

Hm, yes. Grepping shows all uses of this pattern have skip=True, errors='ignore' so perhaps this was used exactly where it should skip the task if no match.

5

Some more debugging. I compared the use of the above with the use of with_first_found, and found that the paths are roughly equivalent, except the lookup plugin has a literal None instead of files in four of the eight places.