# Nested Jinja2 templates and template\_module

**URL:** <https://forum.ansible.com/t/nested-jinja2-templates-and-template-module/7421>\
**Category:** Get Help\
**Tags:** template, playbook, jinja2, ansible-core\
**Created:** [July 23, 2024, 9:23am UTC](https://forum.ansible.com/t/nested-jinja2-templates-and-template-module/7421 "2024-07-23T09:23:09Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![yokodzun](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/yokodzun/32/4280_2.png) [@yokodzun](https://forum.ansible.com/u/yokodzun)\
**Post date:** [July 23, 2024, 9:23am UTC](https://forum.ansible.com/t/nested-jinja2-templates-and-template-module/7421/1 "2024-07-23T09:23:09Z")

</div>

I am using Ansible as part of my GitLab CI/CD pipeline, and I have separate playbooks for rendering configs for my app and Nginx. The folder structure looks like this

```auto
    ├── XXX/YYY/playbook.yml
    ├── templates
    │ ├── app
    │ │ └── app_template.j2
    │ │ └── common_var.inc
    │ │ 
    │ └── nginx
    │ └── sites-available
    │ └── nginx_template.j2

```

All works fine until I try to include some vars from `common_var.inc` in the `nginx_template.j2`. No matter what type of path I am using, relative or absolute, I have got the following error:

```auto
jinja2.exceptions.TemplateNotFound: /home/gitlab-runner/builds/inqzetdg/0/configs/ZZZ/templates/app/common_var.inc

```

The file exists and contains a line like this:

```auto
{% set C.domain = "domain.tld" %}

```

Here is a Snippet from the Playbook that I use for rendering NGINX configs:

```auto
- name: Rendering nginx configs
    template:
      src: "{{ lookup('env','CI_PROJECT_DIR') }}/templates/nginx/sites-available/{{ item }}.j2"
      dest: "/etc/nginx/sites-available/{{ item }}.stage"
      force: yes
    with_items: "{{ vhost_list.stdout_lines }}"

```

I added debug for the ‘ansible\_search\_path’ var, and it looks in the following way:

```auto
# search: ['/home/gitlab-runner/builds/inqzetdg/0/configs/ZZZ/XXX/YYY/playbooks', '/home/gitlab-runner/builds/inqzetdg/0/configs/ZZZ/XXX/YYY/playbooks', '/home/gitlab-runner/builds/inqzetdg/0/configs/ZZZ/templates/nginx/sites-available', '/home/gitlab-runner/builds/inqzetdg/0/configs/ZZZ/XXX/YYY/playbooks', '/home/gitlab-runner/builds/inqzetdg/0/configs/ZZZ/templates/nginx/sites-available'] 

```

How can I add `{{ lookup('env','CI_PROJECT_DIR') }}/templates/app` to the `ansible_search_path`?

Why does the absolute path not work?

Thank you!

---

<div class="post-metadata">

**Author:** ![bvitnik](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/bvitnik/32/4132_2.png) [@bvitnik](https://forum.ansible.com/u/bvitnik)\
**Post date:** [July 23, 2024, 2:25pm UTC](https://forum.ansible.com/t/nested-jinja2-templates-and-template-module/7421/2 "2024-07-23T14:25:11Z")

</div>

How and where did you include `common_var.inc`. That part is not shown.

Generally if you run Ansible with `-vvvv` it will show you all the paths it searched the file in.

---

<div class="post-metadata">

**Author:** ![utoddl](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/utoddl/32/249_2.png) [@utoddl](https://forum.ansible.com/u/utoddl)\
**Post date:** [July 23, 2024, 2:53pm UTC](https://forum.ansible.com/t/nested-jinja2-templates-and-template-module/7421/3 "2024-07-23T14:53:54Z")

</div>

Welcome to the forum, @yokodzun.

I tried reproducing your error locally with the following directory structure and value for `CI_PROJECT_DIR`:

```txt
utoddl@tango:~/ansible/yokodzun$ echo $CI_PROJECT_DIR 
/home/utoddl/ansible/yokodzun
utoddl@tango:~/ansible/yokodzun$ find `pwd` -type f | sort
/home/utoddl/ansible/yokodzun/templates/app/app_template.j2
/home/utoddl/ansible/yokodzun/templates/app/common_var.inc
/home/utoddl/ansible/yokodzun/templates/nginx/sites-available/nginx_template.j2
/home/utoddl/ansible/yokodzun/XXX/YYY/playbook.yml

```

You didn’t show how you were trying to include `app/common_var.inc` from within `nginx_template.j2`, so I attempted it with

```txt
{% include "app/common_var.inc" %}

```

Regardless of how I spelled the path in that include, I was not able to find a working incantation. However, I was able to make it work in two different ways.

The first solution is to create a symbolic link to your `templates` directory next to your playbook(s):

```txt
ln -s ../../templates XXX/YYY/templates

```

The second solution is to move your playbook up two levels to the root of your project – i.e. where Ansible expects it to be.

If you do either of those things, then you can get rid of the “`{{ lookup('env','CI_PROJECT_DIR') }}/templates/`” part of your `src:` parameter and use relative paths for all your `templates`. (And `files`. And `tasks`. And `roles`.)

Ansible uses lots of different PATHs for different things (see “`ansible-config list | grep PATH`”), but playbooks and templates aren’t two of them. (See “`ansible-config list | grep -i PATH | grep -i 'playbook\|template'`”.) Ansible searches for templates relative to the file containing the current play. Yet somehow people really want to bury their playbooks in some directory other than the root of their projects. That’s their choice of course, but they should expect to bear the consequences. I don’t see any up-sides to such non-standard project structures.

---

<div class="post-metadata">

**Author:** ![yokodzun](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/yokodzun/32/4280_2.png) [@yokodzun](https://forum.ansible.com/u/yokodzun)\
**Post date:** [July 24, 2024, 8:50am UTC](https://forum.ansible.com/t/nested-jinja2-templates-and-template-module/7421/4 "2024-07-24T08:50:00Z")

</div>

> How and where did you include common\_var.inc. That part is not shown.

I have been trying on different ways:

```auto
{% from project + '.inc' import C %}
{% from lookup('env','CI_PROJECT_DIR') + '/templates/app/' + project + '.inc' import C %}
{% include 'ZZZ/templates/app/' + project + '.inc' %}
{% include 'templates/app/' + project + '.inc' %}

```

---

<div class="post-metadata">

**Author:** ![yokodzun](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/yokodzun/32/4280_2.png) [@yokodzun](https://forum.ansible.com/u/yokodzun)\
**Post date:** [July 24, 2024, 10:33am UTC](https://forum.ansible.com/t/nested-jinja2-templates-and-template-module/7421/5 "2024-07-24T10:33:09Z")

</div>

> [@utoddl](#):
>
> I don’t see any up-sides to such non-standard project structures.

We have a few different apps with configs stored in Git. The GitLab runner automatically updates those configs via Ansible playbooks.

```shell
ansible-playbook -b --user deploy-user --inventory XXX/YYY/inventories/bla-bla XXX/YYY/playbooks/deploy-config-for-nginx.yml

```

All playbooks are common for all apps, but each app’s configs are stored in its own git repo.  
To avoid copying playbooks to each config repo, we do a ‘git clone’ of the playbook repo to the config repo as the first job of the pipeline.

```auto
- git clone https://gitlab-ci-token:${CI_JOB_TOKEN}@git.tld/team/XXX.git

```

So, `XXX/YYY/playbook.yml` is part of the playbooks repo, and `templates/*` is part of the other.

This is the reason why we have this non-standard structure.

---

<div class="post-metadata">

**Author:** ![Denney-tech](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/denney-tech/32/1556_2.png) [@Denney-tech](https://forum.ansible.com/u/Denney-tech)\
**Post date:** [July 24, 2024, 2:46pm UTC](https://forum.ansible.com/t/nested-jinja2-templates-and-template-module/7421/6 "2024-07-24T14:46:27Z")

</div>

So there’s quite a few problems to tackle with you’re approach.

Firstly, I would attempt to build your common “XXX” repo into an Ansible Collection (even if it only contains roles and playbooks). All of the re-usable code should be in roles, and the playbooks should only exist to call those roles. Any templates found in these roles should be sourced with relative pathing.

Next, in your git CI jobs, you would _install_ this collection from git via ansible-galaxy rather than cloning it (or install from Galaxy if you upload it to the public galaxy or setup a private galaxy). This will help enable Ansible to find everything via relative paths that make sense.

This should allow you to run `ansible-playbook my_namespace.my_collection.my_playbook`, and that playbook is simply:

```yaml
- hosts: localhost
  gather_facts: false
  roles:
    - role: my_namespace.my_collection.my_role

```

All of the important tasks should be in the role(s). You can also pass any variables needed as a json blob with `-e '{"use_my_vars": true, "my_custom_var": "foo"}`

Any custom configs or templates that you would like this Ansible role to _read_ should go in a sensible standardized location in the git repo. `.config/ansible/my_role/templates/include.j2` for e.g. and tree this however it makes sense to you.

[Search paths in Ansible](https://docs.ansible.com/ansible/latest/playbook_guide/playbook_pathing.html) won’t pick up the relative paths of the current working directory, as that context can change unexpectedly during execution. So in order to use the custom configs/templates in each repo, you will need to use the absolute path.

In your common jinja template, I would try one of the following:

```auto
{% include (lookup("env", "CI_PROJECT_DIR") ~ "/.config/ansible/my_role/templates/required.j2") %}
{% include (lookup("env", "CI_PROJECT_DIR") ~ "/.config/ansible/my_role/templates/optional.j2") ignore missing %}

```

or

```auto
{{ lookup("template", lookup("env", "CI_PROJECT_DIR") ~ "/.config/ansible/my_role/templates/required.j2") }}
{{ lookup("template", lookup("env", "CI_PROJECT_DIR") ~ "/.config/ansible/my_role/templates/optional.j2") | default("") }}

```

I’m not entirely sure that the jinja2 include directive will find templates outside of its working tree, hence the second example which uses Ansible’s template lookup plugin to render the template instead.

---

<div class="post-metadata">

**Author:** ![bvitnik](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/bvitnik/32/4132_2.png) [@bvitnik](https://forum.ansible.com/u/bvitnik)\
**Post date:** [July 24, 2024, 3:12pm UTC](https://forum.ansible.com/t/nested-jinja2-templates-and-template-module/7421/7 "2024-07-24T15:12:58Z")

</div>

What’s interesting to me is that Jinja2 is complaining about not being able to find:

`/home/gitlab-runner/builds/inqzetdg/0/configs/ZZZ/templates/app/common_var.inc`

Which looks like a proper absolute path. I’m wondering if `common_var.inc` file is present at all in GitLab Runner working dir after it clones the Git repo.

@yokodzun you were saying that you have this split into multiple GIt repos. Is it possible that GitLab Runner did not pull everything needed to run the playbook.

---

<div class="post-metadata">

**Author:** ![yokodzun](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/yokodzun/32/4280_2.png) [@yokodzun](https://forum.ansible.com/u/yokodzun)\
**Post date:** [July 29, 2024, 8:06am UTC](https://forum.ansible.com/t/nested-jinja2-templates-and-template-module/7421/8 "2024-07-29T08:06:23Z")

</div>

> Is it possible that GitLab Runner did not pull everything needed to run the playbook.

I have checked it manually, and all the necessary files are there.

---

<div class="post-metadata">

**Author:** ![bvitnik](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/bvitnik/32/4132_2.png) [@bvitnik](https://forum.ansible.com/u/bvitnik)\
**Post date:** [July 29, 2024, 10:50pm UTC](https://forum.ansible.com/t/nested-jinja2-templates-and-template-module/7421/9 "2024-07-29T22:50:55Z")

</div>

What comes to my mind is that depending on your GitLab Runner setup and/or CI/CD pipeline, maybe the build folder is mounted inside a container where absolute paths are not valid any more.

You can try to create a dummy CI/CD pipeline that will just run “ls -laR” or similar command inside build folder. After that you can investigate the output in runner console logs and see if anything is different than expected or missing.

---

<div class="post-metadata">

**Author:** ![yokodzun](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/yokodzun/32/4280_2.png) [@yokodzun](https://forum.ansible.com/u/yokodzun)\
**Post date:** [August 6, 2024, 7:38am UTC](https://forum.ansible.com/t/nested-jinja2-templates-and-template-module/7421/10 "2024-08-06T07:38:19Z")

</div>

I applied a workaround, which seemed to solve my problem.  
I have combined a file with variables and a template into one temporary file and then rendered the target config from this file. After that, I cleaned up the temporary files. It’s not an elegant solution, but at least it works.