Continuing the discussion from Async task / queue management:
The fantastic solution from @flowerysong in the above referenced Get Help discussion uses an extremely powerful include_tasks technique: tail recursion. However, the documentation for include_tasks contains no hint of the possibility of nor the considerable restrictions required by tail recursion.
Apparently this became possible in Ansible 2.4, but I’m hesitant to rely on such critical but undocumented behavior in production.
Hi,
Recursion is possible in Ansible at minimum i last 4 year (when I start to use it) and can be used in complex use case like example above, but, like all Recursion can be very tricky to debug and Developer a good exit.
Another you case can be the loop of loop of loop based on variable extract ed on second and third loop.
Like in python, in ansible you need to consider at least 2 key points:
A lot of triky solution are not documented by ansible and awx team and I’m not judge them, documentation require a lot of time, revisione and so on.
We’ll need to answer a few fundamental questions before we can get this documented. Here’s my short list:
Question for the Steering Committee: Is this an intended feature that will be carried into the future, or is this merely a side effect of the way import_tasksinclude_tasks is implemented which could disappear in a future release?
Does this work for any other flavors of include_* or import_*? If so, the scope of the question just got a bit creepier. (See what I did there? 
What circumstances prevent tail recursion? I’ve found by trial and error that having a [Edit: The error I incurred, “… maximum recursion depth exceeded”, was due to a variable defined in terms of itself, not that the final vars: on the last task in the included task files’ last step is one such breaker.include_tasks step had a vars: section.] What else?
Other questions? Maybe someone who knows how it’s implemented would have additional insights…
Tail recursion is an “in-the-weeds” topic anyway. How do we provide enough technical detail to cover it while not overwhelming the include_tasks doc? Should it be split between the include_tasks doc and the tips-n-tricks page, or all in one place? Should an example be added the to async: doc (since async job queue management is a practical example of its application)?
Thanks for the clarification @utoddl!
So I wonder if the @Core team can pipe in here to verify that this is intentional (the answer to question 0 above)…
hi @utoddl
I seen no reason for Ansible to preemptively restrict that use of include_task. However, I must say that using Ansible as a programming language is not something I would be very keen about. To solve your problem, I would try and look into using strategy as @tanganellilore recommended in Async task / queue management - #2 by tanganellilore
As of the documentation for include_task I reckon that recursion (whether it’s tail or not) could be added, but IMHO it should come with a big warning saying that although this is not restricted, one should be extremely careful doing that.
Just my $0.02
You cannot do this type of recursive include with import_tasks, so I’m going to assume that’s a typo. Being able to write something structured this way is a consequence of the existence of include_tasks, not a side effect.
Includes are a documented feature, and changing how they work would be a major breaking change. If you’re unwilling to use features because they could change in the future, technically any feature could disappear or be changed in a future release.
Yes, include_role can also be used recursively.
There are no circumstances that prevent it other than exhausting resources (e.g. hitting Python’s recursion limit or running out of memory.) There are ways to write broken logic that doesn’t work the way you want it to, but that’s not a “circumstance”, that’s broken logic.
- debug:
msg: "{{ foo }}"
- set_fact:
depth: "{{ depth | default(0) + 1 }}"
- include_tasks:
file: recurs.yml
vars:
foo: baz
when: depth < 3
PLAY [localhost] ***************************************************************
TASK [ansible.builtin.include_tasks] *******************************************
included: /home/ec2-user/testing/recurs.yml for localhost
TASK [debug] *******************************************************************
ok: [localhost] =>
msg: bar
TASK [set_fact] ****************************************************************
ok: [localhost]
TASK [include_tasks] ***********************************************************
included: /home/ec2-user/testing/recurs.yml for localhost
TASK [debug] *******************************************************************
ok: [localhost] =>
msg: baz
TASK [set_fact] ****************************************************************
ok: [localhost]
TASK [include_tasks] ***********************************************************
included: /home/ec2-user/testing/recurs.yml for localhost
TASK [debug] *******************************************************************
ok: [localhost] =>
msg: baz
TASK [set_fact] ****************************************************************
ok: [localhost]
TASK [include_tasks] ***********************************************************
skipping: [localhost]
PLAY RECAP *********************************************************************
localhost : ok=9 changed=0 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0
Yes, it was. I meant include_tasks. I’ve edited that post to indicate such. Thanks for catching that.
In that case, it should be sufficient then to document how it can be used to achieve the same effect as until: would have on the initial include_tasks step if it were available.
Likewise for include_role.
My other concerns are moot. Thanks for the perspective.
I’m less sure whether we should change documentation here. While it seems it works, it also seems by one of the posts above that it’s tricky and not recommended (and thus shouldn’t be added to the docs)…
I find the expressed concerns about trickyness amusing and entirely overblown. It’s nothing like so tricky as a Jinja2 expression with several pipes, or why you might choose an include_tasks with a vars: section over a set_fact (lazy evaluation later vs. current static values), or the behavior of tags between imports vs. includes, or the distinctions among all the different types of plugins, or strategies, or variable precedence… Without the tricky bits there wouldn’t be much documentation left!
It is precisely as tricky – no more, no less – than the task level until: keyword, which is not supported on include_* tasks for technical reasons. The user can achieve the exact same effect by adding a self-including include_tasks at the end of the included file that has a “when: not (<condition we would have put on the 'until:' had it been supported>)” on it.
[We could have a separate reasonable discussion about implementing until: on includes by effectively adding such a task onto the end of included files, but that’s for another time.]
The arguments in favor of documenting it are: (1) it’s actually a pretty clean, not-at-all tricky workaround for the lack of until: support on includes, (2) it solves real-world problems (otherwise why have until: at all?), and (3) it’s all but undiscoverable from the current documentation.
I’ve been kicking Ansible’s tires for almost 7 years. My mind was blown last week when I was shown this technique. That hasn’t happened in a while. It shouldn’t be surprising to someone who has read the documents as much as I have. This should be documented.
First, tail recursion was not a ‘designed’ feature, it just happens when you allow recursive inclusion and combine that with how variables work in Ansible. That said, I’m not aware of any plans to remove this feature and we’ve known of it for a long time in core, we just didn’t feel like it was something that should be featured in documentation as that would encourage it’s usage. I’m also not aware of any explicit testing to prevent it from happening accidentally. I would advise you ensure these tests exist in core before you commit to this path, then feel free to document it.
As to why we didn’t want to encourage it’s usage, note that includes are pretty resource intensive, create a serialization of the hosts/tasks execution and have a recursion limit, which we’ve expanded it several times, so it is probable you crash for resource starvation than hitting it now. This is why we normally recommend using imports unless you reaaally need the dynamic nature of an include.
Ansible is designed for automation, not as a full programming language, yes, it can be used as such, but then don’t complain to the hammer manufacturer that the screws end up stripped.
Related proposal that would make most keywords work with includes (including until) 'taskify' includes · Issue #136 · ansible/proposals · GitHub. There have also been efforts to deprecate until/retry as a 2nd loop and integrate it into the ‘loop’ structure [WIP] loop_control until feature by bcoca · Pull Request #62151 · ansible/ansible · GitHub.
Thanks for these insights, @bcoca . Lots to unpack.
[emphasis added] and
and
This inspired me to go back and do some more thorough testing. The results are consistent with the comments above, and sobering.
I owe everyone a huge apology for wasting your time.
Includes do support recursion, but there is no optimization on tail calls. Further, I had been using the term “tail recursion” incorrectly, assuming tail call optimization was implied. It is not.
Using include file tail recursion as a workaround for the missing “until:” support on includes is a non-starter. It may appear to work at first, but it blows its stack at about 250 or so iterations (empirically determined on core 2.14.11, subject to change, etc.).
I am truly sorry, on about 250 levels.
No need to apologize @utoddl ! If you had the questions/perceptions, guaranteed others have and will have in the future. Now we have a topic that covers it in detail and allows others to learn. Great discussion imo.
@utoddl not a waste of time, even if it was … it would be the fault of those that did not document how this works and why you should not do it (me).
This is still a useful ‘trick’ for some hairy situations, but as you saw, must be used with limits and care or things explode.
I’m tempted to add a ‘here be dragons’ section to the docs and start documenting this and other ‘feature traps’ that exist in Ansible. At this point too many intelligent and curious people use it as to be able to hide/ignore this kind of thing. Worse yet, they might start using w/o knowing the reason why we ignored/kept quiet about it.
Anytime subtle tricks are exposed, you’ll find the curious making use of them.
I know I have. 
I suspect that is why I was hired by Ansible inc … to stop exposing such tricks as an outsider and help preemptively patch them as an insider …