# Embedding metadata in yaml files which ansible ignores **URL:** \ **Category:** Get Help\ **Tags:** dev, ansible-core\ **Created:** [April 11, 2024, 7:48pm UTC](https://forum.ansible.com/t/embedding-metadata-in-yaml-files-which-ansible-ignores/4910 "2024-04-11T19:48:53Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![gardar](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/gardar/32/2878_2.png) [@gardar](https://forum.ansible.com/u/gardar)\ **Post date:** [April 11, 2024, 7:48pm UTC](https://forum.ansible.com/t/embedding-metadata-in-yaml-files-which-ansible-ignores/4910/1 "2024-04-11T19:48:53Z")
Hello Ansible Community, I’m currently working on integrating Ansible with other tools and need to embed metadata directly in YAML files (such as `host_vars` and `group_vars`) that Ansible processes. The ideal solution would ensure Ansible completely ignores this metadata, treating it as invisible. **Approaches I’ve Explored:** - **YAML Document Separator (`---`)** Unfortunately, using `---` to separate metadata from variables broke Ansible’s parsing of the YAML file. - **YAML Anchors:** I tried defining an anchor (`&my_metadata:`) and skipping references to it, but Ansible still loaded the data with a “null” key. - **`_ansible_` Prefix:** Inspecting the Ansible source code, particularly `clean.py`, I found out that it strips away variables starting with `_ansible_`. Leveraging this, I experimented with using a variable like `_ansible_my_metadata_` to store metadata. It seems Ansible ignores this variable—no errors are thrown, and it does not appear when I print all variables with `{{ vars }}`. - **Comments:** I considered using comments for storing the metadata but parsing them is not ideal, as the external tool I’m using leverages PyYAML, which struggles with it. - **Obscure Keys:** I also considered using obscure keys for my metadata, but there’s a risk that Ansible could potentially use or process the metadata, which I wish to avoid. - **Separate Files:** While storing metadata separately works, I was hoping to keep everything within the Ansible YAML structure for enhanced portability. **Questions for the Community:** 1. Are there any other methods for embedding metadata that Ansible will reliably ignore? 2. Are there potential drawbacks to relying on the `_ansible_` prefix behavior that I should be aware of? ---
**Author:** ![Denney-tech](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/denney-tech/32/1556_2.png) [@Denney-tech](https://forum.ansible.com/u/Denney-tech)\ **Post date:** [April 11, 2024, 7:57pm UTC](https://forum.ansible.com/t/embedding-metadata-in-yaml-files-which-ansible-ignores/4910/2 "2024-04-11T19:57:25Z")
Welcome to the forum @gardar ! Can you expand a little more on what this metadata is intended for? Why would you want host\_vars/group\_vars to exist, but be ignored by Ansible? When and where do you want this data to be used, and by what if not Ansible? My first thought on how you could define your metadata in an _ansible_ way that isn’t immediately consumed, is to define them in a role. Then that metadata is only available when you call the role, not when the inventory is initialized. Otherwise you could have vars directory in the root of your project with the metadata there as a vars file. Edit: Honestly, you could just put it anywhere outside of inventory, and tell your tool where to find it and it should work. `./vars/meta.yml` `./.meta.yml` `./.toolname.meta.yml` or something along these lines would work. Ansible doesn’t consume vars files outside of its inventory unless it’s in a role’s defaults or if you explicitly `include_vars:` files. ---
**Author:** ![gardar](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/gardar/32/2878_2.png) [@gardar](https://forum.ansible.com/u/gardar)\ **Post date:** [April 12, 2024, 1:34pm UTC](https://forum.ansible.com/t/embedding-metadata-in-yaml-files-which-ansible-ignores/4910/3 "2024-04-12T13:34:11Z")
Thanks for the reply, let me provide more context on the specific use case and why I am exploring these particular options. I’ve developed a script that interfaces with the Red Hat Satellite/Foreman API to export Ansible variable overrides and convert them into the appropriate Ansible `host_vars` and `group_vars` format. In Satellite/Foreman, Ansible variables are managed differently than in the standard Ansible CLI environment. Variables in Satellite/Foreman can have override matchers that determine their application scope. For instance: - To apply an Ansible variable to a specific host group (akin to placing it under `group_vars/mygroup.yml` in Ansible), you would set an override matcher in Satellite/Foreman like `hostgroup = mygroup`. - To target a variable to a specific host (`host_vars/myhost.yml`), you define an override matcher such as `fqdn = myhost`. Moreover, it’s possible to define additional matchers based on attributes like operating system, domain, location, subnet, etc. My script extracts these variables and their corresponding matchers from Satellite/Foreman and structures them into the format that Ansible can understand (`host_vars`, `group_vars`). The next phase of my project involves importing these variables back into Satellite/Foreman. However, due to the differences in how variables are handled between Satellite/Foreman and Ansible, I need to include additional details about each variable within the `host_vars` and `group_vars` files. These details are crucial for correctly placing the variables back into Satellite/Foreman. My goal is to embed this additional metadata within the Ansible variable files in a way that Ansible itself will ignore—since there’s no need for Ansible to process this metadata, and to prevent any potential issues from Ansible trying to use this extra information. Your suggestion to use a separate directory for metadata is certainly a feasible solution, and I’m considering it as a backup plan. However, ideally, I would prefer to keep everything integrated within the existing Ansible structure for simplicity and portability reasons. Hence, I’m exploring if there’s a way to make this metadata invisible or ignored by Ansible while still being stored alongside the relevant Ansible data. I hope this clarifies my intentions and the challenges I’m facing. ---
**Author:** ![chris](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/chris/32/293_2.png) [@chris](https://forum.ansible.com/u/chris)\ **Post date:** [April 12, 2024, 1:41pm UTC](https://forum.ansible.com/t/embedding-metadata-in-yaml-files-which-ansible-ignores/4910/4 "2024-04-12T13:41:31Z")
Could you comment the metadata? ---
**Author:** ![gardar](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/gardar/32/2878_2.png) [@gardar](https://forum.ansible.com/u/gardar)\ **Post date:** [April 12, 2024, 1:49pm UTC](https://forum.ansible.com/t/embedding-metadata-in-yaml-files-which-ansible-ignores/4910/5 "2024-04-12T13:49:57Z")
Thanks for the idea! Unfortunately, using comments isn’t ideal because PyYAML, which I’m using to parse the YAML, doesn’t keep the comments—it just ignores them [Load comments · Issue #90 · yaml/pyyaml · GitHub](https://github.com/yaml/pyyaml/issues/90). I’d need to come up with a workaround to handle the comments separately, which feels a bit clunky and hacky. I was hoping there would be a more straightforward way to do this. ---
**Author:** ![Denney-tech](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/denney-tech/32/1556_2.png) [@Denney-tech](https://forum.ansible.com/u/Denney-tech)\ **Post date:** [April 12, 2024, 1:50pm UTC](https://forum.ansible.com/t/embedding-metadata-in-yaml-files-which-ansible-ignores/4910/6 "2024-04-12T13:50:23Z")
Wow! Thank you for the in-depth explanation of your objective. In regards to the initial issue and what you’re trying to do, even though you’re trying to structure Satellite/Foreman inventory vars in an Ansible inventory like manner, you’re wanting to preserve how it appears in Satellite/Foreman and be able to post changes back where they came from. I suggest that you try _not_ to emulate Ansible inventory structure, because it isn’t from Ansible inventory. You should try and maintain parity with Satellite/Foreman, so that you don’t get confused while transposing between the two, _and_ you won’t turn them into Ansible inventory vars that get consumed automatically. Something like: foreman\_vars/parameters.yml ```auto --- hostgroups: - name: group1 vars: - name: var_name value: var_value type: hosts: - name: host1 vars: - name: var_name value: var_value type: orgs: - name: default_org vars: - name: var_name value: var_value type: ... ``` etc. At least, that’s how I would approach it. You might even be able to turn this into a jinja2 template if you wanted to define parameters in Satellite/Foreman as code. That said, there’s a lot of tooling out there already for these sorts of things. There’s the redhat.satellite/[theforeman.foreman](https://galaxy.ansible.com/ui/repo/published/theforeman/foreman/docs/) and redhat.satellite\_operations/[theforeman.operations](https://galaxy.ansible.com/ui/repo/published/theforeman/operations/) collections, which contain _roles_ for building Satellite/Foreman as code. ---
**Author:** ![gardar](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/gardar/32/2878_2.png) [@gardar](https://forum.ansible.com/u/gardar)\ **Post date:** [April 12, 2024, 2:19pm UTC](https://forum.ansible.com/t/embedding-metadata-in-yaml-files-which-ansible-ignores/4910/7 "2024-04-12T14:19:32Z")
It would definitely make life more easy if I just wanted to make a backup of the variables from Satellite/Foreman. However my goal is twofold. - I want to be able to have the variables in Satellite/Foreman backed up and stored in a git repo, as Satellite/Foreman doesn’t have any version control for ansible variables and it’s super easy to accidentally delete variable content in Satellite/Foreman. - I want to be able to consume the variables with ansible as host/group vars, so that I can execute ansible outside of Satellite/Foreman with the same variables as in Satellite/Foreman. There’s actually a [theforeman.foreman.foreman](https://docs.ansible.com/ansible/latest/collections/theforeman/foreman/foreman_inventory.html) inventory plugin available, in my script I’ve matched the host group structure that the inventory plugin has so that I can currently use my exported host\_vars and group\_vars files with the foreman inventory. ---
**Author:** ![Denney-tech](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/denney-tech/32/1556_2.png) [@Denney-tech](https://forum.ansible.com/u/Denney-tech)\ **Post date:** [April 12, 2024, 2:22pm UTC](https://forum.ansible.com/t/embedding-metadata-in-yaml-files-which-ansible-ignores/4910/8 "2024-04-12T14:22:23Z")
The vars can be exported by theforeman inventory plugin as part of the dynamic inventory, so your hosts will consume and inherit vars exactly as they are defined in theforeman. ---
**Author:** ![gardar](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/gardar/32/2878_2.png) [@gardar](https://forum.ansible.com/u/gardar)\ **Post date:** [April 12, 2024, 2:25pm UTC](https://forum.ansible.com/t/embedding-metadata-in-yaml-files-which-ansible-ignores/4910/9 "2024-04-12T14:25:45Z")
Absolutely, exporting the variables or using them from the inventory plugin is no problem, the challenge is converting the vars from yaml host/group vars back to satellite/foreman 🙂 ---
**Author:** ![Denney-tech](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/denney-tech/32/1556_2.png) [@Denney-tech](https://forum.ansible.com/u/Denney-tech)\ **Post date:** [April 12, 2024, 2:29pm UTC](https://forum.ansible.com/t/embedding-metadata-in-yaml-files-which-ansible-ignores/4910/10 "2024-04-12T14:29:26Z")
Right, so there’s no need to reinvent the wheel. Script your exported vars in a way that’s easy for you to work with when importing it back to satellite/theforeman (potentially through the existing roles), but let the inventory plugin worry about consuming it when you actually want it in your inventory. Don’t try to layer static inventory vars on top of the dynamic inventory.