I’m curious if the 2.19 templating changes might affect (in a good way; fingers crossed) our chances of someday being able to use macro/endmacro in useful ways in Ansible templates. They work as expected in template files, but in templated string values, not so much. Behold:
If you squint you might notice that both foodef_alpha and bardef contain the definitions of otherwise identical macros named foo and bar. foodef_alpha also contains a templated expression that uses its macro definition successfully. That is, stuff.my_alpha ends up with the value of A.
What doesn’t work is the bardef macro definition without an invocation. It doesn’t “fail” in the Ansible task sense. It just doesn’t accomplish anything. As far as I’ve found there’s no way to define a macro in one string and invoke it in others. (Exception: included files, as per normal Jinja usage. But this is about strings defined in Ansible vars etc.)
This has been touched on a couple of times before. The TL;DR boils down to, “It’s complicated, because of the way templates are done.” Fair enough. But seeing how templating is getting a makeover, it seems reasonable to ask about it again.
The primary problem is one of scoping- while playbooks can contain Jinja templates and expressions, the playbook itself is not a Jinja template and doesn’t execute under a Jinja scope (as anyone who’s tried to affect playbook shapes with Jinja has discovered the hard way). Most of our Jinja integration lives in the variable subsystem- the numerous ephemeral Jinja scopes we use are created dynamically beneath each host/task/loop-item and only live for the duration of rendering a single template. So there’s not currently a supported mechanism to preserve Jinja locals (like macros), since that stuff all disappears with the Jinja context.
I’m kinda burying the lede here, but several of the templating engine improvements in 2.19 actually allow what you’re doing to work now. You just need a tiny tweak to your assignment to an Ansible variable and Bob’s Your Uncle:
vars:
# ...
# note that the template result is not the macro block itself or an invocation of the macro, but the generated macro callable from the Jinja local `bar`
bardef: '{% macro bar(key_) %}{{ ddd[key_] | default(key_) }}{% endmacro %}{{ bar }}'
tasks:
- debug:
var: bardef("beta") # woot, works in 2.19.0b5
- debug:
var: bardef("boguskey") # doh, broken in 2.19.0b5
The default fallback case in your sample actually exposed a small bug in the datatag access mechanism that prematurely halts expression eval when an error occurs outside a filter/test/operator invocation- I’ve fixed that locally, so with that fix applied, both cases work correctly:
This works, but it’s somewhat inefficient, since we don’t currently allow storage of callables (eg via set_fact), so it actually has to recompile the macro on every invocation. I’ve got a compiled template cache sitting on a shelf that bypasses all that, but it’s not ready for prime-time.
Thanks for the testing and feedback! I’m going to polish up the access fix and add some explicit test coverage for this use case, but seems like between the bugfix (which should be available in beta6) and the template tweak to return the macro callable, this gives you what you need, yeah?
Wow! That’s fantastic. I felt kind of bad about bringing it up again, but if it helped you work out a corner case bug, that makes it worth it. That it will solve my problem is, frankly, an unexpected bonus.
When last discussed (links up-thread), there was talk of “collection scoped macro plugins”. Like maybe that would be the clean way to expose Jinja macros in template strings. Still a neat idea - scoping just feels somehow less fragile.
Now that I see how you got this to work - without scoping - um, hm. Care to speculate on collection scoped macro plugins ever becoming a thing?
(This is all fascinating to me, lifting the corners and peeking at these implementation details. Ansible has grown up so much in the last decade!)
I wanted to address this point separately. It makes sense that this would impose some costs. However, the alternative is to duplicate the macro implementation logic - and keep those copies in sync - everywhere we would otherwise call mymacro('param'). Wouldn’t that incur the same efficiency penalties? Let the machines work for us, I say.
It makes sense that this would impose some costs. However, the alternative is to duplicate the macro implementation logic
Oh yeah, that wasn’t an indictment of its use- it still works WAY better as a macro like this than inlined all over the place, so I’m thrilled that we “accidentally” fixed it in 2.19 . I was mainly noting that there are several things we could do to further increase its efficiency under the covers. The lazy templating and intra-operation indirect template caching we’ve already added with data tagging in 2.19 should make its use in loops much cheaper than it would’ve been in older releases (had the other things blocking it from working ever been fixed).
Now that I see how you got this to work - without scoping - um, hm. Care to speculate on collection scoped macro plugins ever becoming a thing?
I’m still not opposed to the idea, but TBH (and especially with this working as a stop-gap), it’s unlikely to be a priority for at least the next couple releases. I recently figured out a couple other low-hanging-fruit templating tricks that make it much easier to do arbitrary-expression list/dict comprehensions entirely within Jinja + Ansible, but it’s pretty late to try and slam those in for 2.19, so it’ll probably be 2.20…
Hi. I’ve just started to test some of my macros with ansible-core 2.19+ and found out that output of the macro is always considered to be a string even when it is a list or dict. This completely breaks my usecase. Here are some repros:
Of course, my macros are much more complex and they calculate some complex variable based on some input. This prevents usage of macros as simple functions.
So is this a bug, a deliberate change or something else? Is there a workaround? Piping to from_json also does not work because from_json expects double quotes to be used.
How about | from_yaml instead of | from_json ? I don’t have a 2.19 handy to try it on, but on 2.18 from_yaml doesn’t complain. (But 2.18 doesn’t show the problem anyway.) This “extended json” is valid yaml after all.
There’s a newline after the content, so it’s not just the value you want to return, but also a string with a newline in it, and Ansible then concatenates these results together.
@utoddlfrom_yaml works. Thanks. | replace("'", '"') | from_json also works.
@felixfontein OMG . Why the newline makes the problem all of a sudden. Can we consider this to be a bug/regression? Thanks alot . I’m using YAML output so I never saw that extra newline character.
But in <= 2.18, either newline is not present in the output or it is ignored. Macro output is processed as proper data type. Anyway, behavior is changed from 2.18 → 2.19 and macro examples I gave look correct and should behave correctly even without removing extra whitespace (newline)… or am I missing a bigger picture?
In 2.18 the newline is present in the output, but ansible-core parses the output again (and if it can’t parse it, it just uses the string). This was the behavior which is finally gone and which was causing so many problems in the past, I don’t think it makes sense to return to it.
Hmmmm, OK. Makes sense. But what now? Should this be noted in porting documentation? Or should macro output be handled in special (the old) way? I’m not the first nor the last person that will try to use macros this way and then fail for not so obvious reason.
Can you expand on that a bit, @felixfontein ? It seem like in the past (through 2.18) that if the resolved value looked like valid json or yaml, then it would be treated as such. But if the resolved value is not parsed again by ansible-core, what does it do instead? That is, how does a string that looks like yaml with or without a trailing new-line end up becoming or not becoming something other than a plain string?
Not if it “looked like valid json or yaml”, if it was a valid string representation of a Python data structure. Templating {{ ['bar'] }} would return the string ['bar'], which Ansible would have to parse back into a data structure. Now it returns the actual data structure.
Great shout-out to @nitzmahone for the work he has done on making macros finally great in Ansible. Just seen the presentation today at CfgMgmtCamp 2026 (online for me, unfortunately) and was really surprised. I obviously did not pay close attention to this thread and to this line in particular:
but when Matt said “… it’s returning the macro function” during the presentation it just blew my mind… and everything became clear. It felt like something I’ve waited for years and finally got. I’ve tried several different approaches in making macros first class citizens in my code, including the way to distribute them, but it always felt reeeeaaaly ugly. This way of defining and using macros is way more practical. The only downside is that it is not compatible with versions of Ansible older than 2.19.
Anyway, you can check the whole presentation here:
I’ve asked @nitzmahone after the presentation. And turns out in ansible-core 2.19 it is possible to return macro from lookup plugin. This might be one way to distribute macros as part of ansible collection.
Continuing with present example it would look something like this:
@kks well, in >=2.19, distributing macros should be a “non issue” since you can define a macro in a variable. I haven’t still tested but Matt’s example suggests that you can just define macros as role defaults (defaults/main.yml) or role variable (vars/main.yml). You can then create a dummy role with no tasks and just import it or make other roles dependent on it and you will gain access to all macros defined in the role. Role can be part of the collection, so that solves everything.
My latest attempt to distribute macros was exactly by using a dummy role just like that. Unfortunately, I was not able to avoid using {% import %} statements so it was “a must” in any place I wanted to use a macro. Macros themselves were specified in .j2 template files in the role. The hard limitation I’ve hit was Jinja’s security requirement that importable templates MUST reside inside your Ansible project (Ansible search path). This works just fine if your collections are installed inside your Ansible project (which is a good practice any way). I just point the import to a proper path like:
{% from "collections/ansible_collections/bvitnik/mycol/roles/common/templates/macros.j2" import some_macro %}
and it works.
The problem arises if you install your collections globally, e.g. ~/.ansible/collections. Import does not work at all, either with relative or absolute path to the template file containing macros.
Making it possible to define macros in variables, makes this problem go away.
)
. I was mainly noting that there are several things we could do to further increase its efficiency under the covers. The lazy templating and intra-operation indirect template caching we’ve already added with data tagging in 2.19 should make its use in loops much cheaper than it would’ve been in older releases (had the other things blocking it from working ever been fixed).
. Why the newline makes the problem all of a sudden. Can we consider this to be a bug/regression? Thanks alot 
. I’m using YAML output so I never saw that extra newline character.