Task using with_fileglob shows warnigs when skipped

Get Help
, ,
1

I’m running into a somewhat irritating issue and would like to ask for feedback or help on this.

We have to install / update a helm chart on kubernetes that includes some CRDs. But helm doesn’t update CRDs. And we want to keep them up-to-date. Actually, we recently ran into some problems because they weren’t.

So my solution is to “install” / “update” the chart with kubernetes.core.helm and check_mode: true. If, and only if, this would install or update the chart I run a block of tasks that:

  1. unpack the chart to temp_directory/chart
  2. apply all CRD YAML files with kubernetes.core.k8s using with_fileglob: temp_directory/chart/crds/*.yaml
  3. install / update the chart

If not, the block is skipped.

But when there’s no installation or update needed, I get a warning that there are no temp_directory/chart/crds/*.yaml files. Of course there aren’t, because we didn’t unpack the chart. But since this task is skipped, anyway, why the warning?

What’s the reason for this? And, if there’s a reason for this, is there a way to suppress this warning for a task that is skipped and only then? I’d still like to see warnings if the task isn’t skipped.

BTW: I think this has to do with ansible-core and not the kubernetes.core collection. I’ve tagged this with kubernetes, anyway, because maybe someone using ansible to manage helm charts with CRDs might have a better solution than the one that I thought up.

2

Having not provided any output or reproducer, I can only imagine it is the common “gotcha” that when conditionals run for each iteration of a loop, and not before the loop.

I will note that I don’t know what warning you are getting. with_fileglob doesn’t produce any warnings itself when nothing exists, unless the directory where the *.yaml files are suppose to exist doesn’t.

If that is the case, a few ideas:

  1. Use the file module to ensure the parent dir exists
  2. Use include_tasks with a when instead of when statements on individual subsequent tasks.
3

Run them as a task file? As was mentioned, `when` runs for all cycle iterations.

- name: Check if installation is required
  ....
  register: installation_required

- name: Run the installation (if required)
  ansible.builtin.include_tasks:
     file: "{{ installation_required | ternary('install.yml', 'noop.yml') }}"

Where noop.yaml is an empty but existing file. Noop - no operation.

4

Well it’s basically:

- name: Check installed
  kubernetes.core.helm:
    host: https://{{ inventory_hostname }}.{{ dns_zone }}:8443
    context: "{{ inventory_hostname }}"
    kubeconfig: "{{ playbook_dir }}/tmp/config-{{ inventory_hostname }}"
    validate_certs: true
    name: myapp
    chart_ref: "oci://{{ my_registry }}/myapp/mychart"
    release_namespace: myapp
    chart_version: "{{ chart_version }}"
  check_mode: true
  register: install_result

- name: Install / update myapp
  when: install_result.changed
  block:
    - name: Get chart
      kubernetes.core.helm_pull:
        chart_ref: "oci://{{ my_registry }}/myapp/mychart"
        chart_version: "{{ chart_version }}"
        destination: "{{ playbook_dir }}/tmp/{{ inventory_hostname }}"
        untar_chart: true
      check_mode: false

    - name: Install / update myapp CRDs
      kubernetes.core.k8s:
        state: present
        src: "{{ item }}"
      with_fileglob:
        - "{{ playbook_dir }}/tmp/{{ inventory_hostname }}/mychart/crds/*.yaml"
        - "{{ playbook_dir }}/tmp/{{ inventory_hostname }}/mychart/crds/*.yml"

If the chart is up to date, the block is skipped. But nevertheless I get this warning:

TASK [Install / update myapp CRDs] *******************************************************************************************
[WARNING]: Unable to find '/home/mariolenz/playbook/tmp/test-cluster-001/mychart/crds' in expected paths (use -vvvvv to see paths)
skipping: [test-cluster-001]
5

BTW this is unsecure. As soon as there are predictable path patterns for temp data - this temp data can be changed after you’ve created it and before you read it.

I wounder how output with -vvvvv looks like.

6

A common and not at all unreasonable misconception. Blocks are never skipped.

A block is a convenient way to apply things — in this case, a when: condition — to each iteration of tasks in a set of tasks.

So your block’s when: condition is evaluated for each iteration of your with_fileglob: list, each of which would generate the warning you see. You only see the warning once because of Ansible’s “suppress identical warnings” functionality.

7

Well, I guess you’re right… but it doesn’t explain why I see a warning on a skipped task. I don’t care for any warnings, it’s skipped anyway.

It’s not really a big issue, just somewhat annoying.

8

As I mentioned above, when conditions are applied for each iteration of a loop, not before the loop is evaluated. So with_fileglob is evaluated before the when.

9

I appreciate your perspective, but the warning is generated before Ansible can determine whether the task should be skipped.

You don’t care about the warning because you already know that path doesn’t exist.
But consider the case of a user running a similar task except for a typo in the with_fileglob path which results in the task being skipped. That user would (rightly) be here on the forum asking why the task is being skipped with no hint that something is amiss!

Far better to emit a warning that some users won’t care about than to omit a warning that costs other users days of hair-pulling frustration.

You bring up and interesting point, though. Deciding what to include in a job log is a delicate balancing act. Job logs need to contain all anomalous behavior relative to a given verbosity level - like your warning - while not overwhelming users with (details×repetition) - hence Ansible’s “suppress identical warnings” functionality for instance.

10

Thanks for the explanation @sivel! As I’ve said it’s not really a big deal, just somehow annoying and irritating. We’ll just just ignore this warning. I just wasn’t able to find anything about this behavior and wanted to satisfy my curiosity.

I think “delicate” is the right word. In my case, this warning doesn’t make sense. But I can think up use cases where showing warnings even if a task is skipped makes perfect sense. As I’ve said, we’ll simply ignore this since we know it’s now an issue for us.

Thanks all!

11

Use with: with a fileglob query (*.ya?ml should probably work) and errors=ignore?

12

That fileglob, *.ya?ml, is unlikely to match any existing files. The “*” part is valid for fileglobs, but “ya?ml” is a regular expression — assuming you meant to make the a optional. Instead, that would match yaxml, yayml, and yazml, but not either of yaml or yml.

The ansible.builtin.fileglob filter will accept more than one glob pattern (see man 7 glob). To match both *.yml and *.yaml, you have to use (at least) two glob patterns. Behold:

utoddl@tango:~/ansible$ ansible localhost -m debug -a \
>     "msg={{ lookup('ansible.builtin.fileglob', \
>                        '/home/utoddl/ansible/*.yml', \
>                        '/home/utoddl/ansible/*.yaml', \
>                            wantlist=true) | length }}"
localhost | SUCCESS => 
    msg: 280
utoddl@tango:~/ansible$ ansible localhost -m debug -a \
>     "msg={{ lookup('ansible.builtin.fileglob', \
>                        '/home/utoddl/ansible/*.ya?ml', \
>                            wantlist=true) | length }}"
localhost | SUCCESS => 
    msg: 0

Cheers.

13

@utoddl I think you’re right, *.ya?ml looks like a regex and not a fileglob and would match neiter *.yml nor *.yaml. I’m not 100% sure if you can have an optional character in a fileglob. However, with_fileglob allows to add a list and I’m perfectly fine with having *.yml and *.yaml as two list items. But if it’s possible to do this with a fileglob in one go or not hasn’t been the question, anyway.

@ildjarn I was looking for a way to see warnings when the task is actually executed, but not if it’s skipped. As far as I understand this isn’t possible ATM because with_fileglob is evaluated (and shows this warning) before the when is evaluated and the task is skipped as @sivel explained above.

As stated above, we’ll just live with the situation. It’s somewhat annoying, but not that bad.

14

Of course this is fairly out of scope, but I’ve been curious. So for the record, *.ya?ml didn’t work for me on Bash but *.y{a,}ml did:

$ touch test.yml
$ touch test.yaml
$ touch test.yaxml
$ ls *.ya?ml # wrong output
test.yaxml
$ ls *.y{a,}ml # correct output
test.yaml  test.yml

Not sure if it works for with_fileglob, though. Anyway, the question wasn’t how to do it in one go instead of having two list item for *.yml and *.yaml.

As I’ve said, this is out of scope here. Just wanted to share my findings about optional characters in fileglobs, at least on Bash. Maybe it helps someone.

15

Using *.y{a,}ml in bash takes advantage of two distinct things: Brace Expansion and Pathname Expansion. They each get their own section in the bash manual (man bash).

Brace expansion happens first. It turns the single term *.y{a,}ml into two distinct terms: *.yaml and *.yml — in that order. Each of those terms goes through several other passes (tilde expansion, parameter and variable expansion, command substitution, arithmetic expansion, and word splitting) before finally getting to pathname expansion. The presence of any of the fileglob metacharacters *, ?, and [ will trigger pathname expansion.

Ansible’s fileglob only does the fileglob part. Some other shells do even more magic than bash such as reordering terms by matching filesystem objects’ timestamps.

But like you said, now we’re way out of scope. It is interesting though, to see how pieces of functionality associated by convention with various magical strings pop up in unrelated technical contexts. It’s a productive thing to be curious about, IMO.