# Molecule: make tests FAIL **URL:** \ **Category:** Get Help\ **Tags:** role, molecule\ **Created:** [July 2, 2025, 4:13pm UTC](https://forum.ansible.com/t/molecule-make-tests-fail/43698 "2025-07-02T16:13:07Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![daks](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/daks/32/5277_2.png) [@daks](https://forum.ansible.com/u/daks)\ **Post date:** [July 2, 2025, 4:13pm UTC](https://forum.ansible.com/t/molecule-make-tests-fail/43698/1 "2025-07-02T16:13:07Z")
I have started creating some roles for my use and implemented some tests using Molecule which now uses Ansible as verifier. I’m not sure where I saw it but I have implemented tests using standard Ansible modules but with `check_mode: true` which is fine but doesn’t makes the `verify` step fail, errors are reported as changes. ```auto # molecule/default/verify.yml - name: Verify backupninja configuration ansible.builtin.lineinfile: path: /etc/backupninja.conf state: present line: "{{ item }}" check_mode: true loop: - '# Ansible managed: Do NOT edit this file manually!' # default config variables - 'reportsuccess = yes' - 'when = everyday at 01:00' # overriden config variables - loglevel = 1 - reportemail = vagrant ``` if I do not provide the correct vars I got the following result ```auto TASK [Verify backupninja configuration] **************************************** [WARNING]: Found variable using reserved name: name ok: [debian-12] => (item=# Ansible managed: Do NOT edit this file manually!) ok: [debian-12] => (item=reportsuccess = yes) ok: [debian-12] => (item=when = everyday at 01:00) changed: [debian-12] => (item=loglevel = 1) changed: [debian-12] => (item=reportemail = vagrant) [...] PLAY RECAP ********************************************************************* debian-12 : ok=4 changed=2 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0 ``` I’m quite sure that the changed result is not what I want in a CI pipeline, I want those tests to report FAILURE. What is the correct solution? _bonus question: what about this `[WARNING]: Found variable using reserved name:` error which I have sometimes (with different variables names)_ ---
**Author:** ![Jeff\_Pullen](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/jeff_pullen/32/5318_2.png) [@Jeff\_Pullen](https://forum.ansible.com/u/Jeff_Pullen)\ **Post date:** [July 2, 2025, 4:39pm UTC](https://forum.ansible.com/t/molecule-make-tests-fail/43698/2 "2025-07-02T16:39:54Z")
I think there might be some misunderstanding of the verify stage. The verify job should check whatever values/settings/state you need and then explicitly fail the job if they aren’t the way you expect. Issue #1 - The lininfile module isn’t going to fail it in the way that you want, regardless of how you set check mode. You could use it to register a variable and then assert the value of that variable later to determine if the job should fail. Issue #2 - Molecule should ideally be permitted to run destructive tests since it provides the capability to create and destroy test systems. Its not clear to me if you’re using check mode to “protect” the test system. If that is the case, I would suggest against it and instead provision tests systems using the create stage. I created an example molecule setup for use in the Red Hat Developer Sandbox. It is a simple role to make a backup of a file and then verify multiple aspects of that operation. You’ll notice that it doesn’t fail the job until the very end so that I can gather up all the failures rather than bailing on the first issue. [Example code can be found here](https://github.com/devspaces-samples/ansible-devspaces-demo/blob/devspaces-3-rhel-8/collections/ansible_collections/sample_namespace/sample_collection/extensions/molecule/default/verify.yml). If you want to test this out yourself, [there is a blog here](https://developers.redhat.com/articles/2023/10/02/boost-ansible-developer-experience-openshift-dev-spaces#) that shows you how to spin up the free environment to experiment with this specific example. ---
**Author:** ![daks](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/daks/32/5277_2.png) [@daks](https://forum.ansible.com/u/daks)\ **Post date:** [July 2, 2025, 4:43pm UTC](https://forum.ansible.com/t/molecule-make-tests-fail/43698/3 "2025-07-02T16:43:57Z")
I know that because it runs in a temporary container or VM, whatever happens in Molecule has no impact. _(I previously used Test-kitchen with Salt and Inspec on Vagrant VM.)_ I used `check_mode` because I think I saw it somewhere in an article or molecule tests. I thought it could be used for that. If I understand your example correctly, you use `ansible.builtin.assert` to test your molecule setup by verifying the variables you ‘register’ using other methods (stats or whatever). Does that mean that in my case I could continue using my `lineinfile` or `template` or other module I currently use by simply registering the result and then testing it with `assert`. For me it looks like writing the tests will be longer, with 2 or 3 steps instead of only one (when I used Inspec) ---
**Author:** ![daks](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/daks/32/5277_2.png) [@daks](https://forum.ansible.com/u/daks)\ **Post date:** [July 2, 2025, 4:54pm UTC](https://forum.ansible.com/t/molecule-make-tests-fail/43698/4 "2025-07-02T16:54:09Z")
> For me it looks like writing the tests will be longer, with 2 or 3 steps instead of only one (when I used Inspec) what I mean by saying that: instead of ```auto whatever_method_to_verify ``` do I need to write ```auto one_verification: register: variable assert: variable is defined and variable is equal to "xx" ``` ? It seems quite verbose to me and I would like to know if there is any way to test/verify in one step. But maybe it’s not possible using Ansible as a verifier. ---
**Author:** ![Thulium-Drake](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/thulium-drake/32/833_2.png) [@Thulium-Drake](https://forum.ansible.com/u/Thulium-Drake)\ **Post date:** [July 2, 2025, 6:44pm UTC](https://forum.ansible.com/t/molecule-make-tests-fail/43698/5 "2025-07-02T18:44:55Z")
Normally I use commands/modules to validate the program’s functionality e.g. - Use `ansible.builtin.uri` to retrieve a webpage and check if the result contains a specific text - Run a command that only functions if the configuration is correct for the testcase - Start a secondary container that will do some remote probing in some way. You can check some of these roles for inspiration 🙂 - [ansible-role-nginx\_revproxy/molecule/default/verify.yml at master · Thulium-Drake/ansible-role-nginx\_revproxy · GitHub](https://github.com/Thulium-Drake/ansible-role-nginx_revproxy/blob/master/molecule/default/verify.yml) - [ansible-role-yum\_repository/molecule/default/verify.yml at master · Thulium-Drake/ansible-role-yum\_repository · GitHub](https://github.com/Thulium-Drake/ansible-role-yum_repository/blob/master/molecule/default/verify.yml) - [ansible-role-dns\_server/molecule/default/verify.yml at master · Thulium-Drake/ansible-role-dns\_server · GitHub](https://github.com/Thulium-Drake/ansible-role-dns_server/blob/master/molecule/default/verify.yml) Using `failed_when` and `ansible.builtin.assert` is basically most you need. You can use it to check values or compare stuff. You could even have (though molecule should already do that for you in the idempotency test) have it fail like this: ```auto - name: 'Put template' ansible.builtin.template: src: 'template.j2' dest: '/etc/file.conf' owner: 'root' group: 'root' mode: '0644' register: file_template failed_when: file_template['changed'] ``` So in a nutshell, because you’re using ‘normal’ tasks/plays to validate the role, you can do anything you want/need! ---
**Author:** ![Jeff\_Pullen](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/jeff_pullen/32/5318_2.png) [@Jeff\_Pullen](https://forum.ansible.com/u/Jeff_Pullen)\ **Post date:** [July 2, 2025, 6:48pm UTC](https://forum.ansible.com/t/molecule-make-tests-fail/43698/6 "2025-07-02T18:48:17Z")
Yes, that is often the way. There are some tests that may not require variables being registered. For example if your role installed a service, you may include a test that just starts the service. In that case you may not need the extra granularity that comes with the register/assert. Your example didn’t include the block rescue, but I’d also encourage the use of that. That way you can capture multiple failures in a single test. It also allows you to include negative/expected failure tests. ---
**Author:** ![daks](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/daks/32/5277_2.png) [@daks](https://forum.ansible.com/u/daks)\ **Post date:** [July 4, 2025, 2:21pm UTC](https://forum.ansible.com/t/molecule-make-tests-fail/43698/7 "2025-07-04T14:21:16Z")
So adding ```auto register: action_result failed_when: action_result['changed'] ``` does the job, thanks! When executed in a loop, it even groups and returns all errors together like ```auto TASK [Verify backupninja actions] ********************************************** failed: [debian-12] (item=10-system.sys) => {"ansible_loop_var": "item", "changed": true, "failed_when_result": true, "item": "10-system.sys"} failed: [debian-12] (item=11-system.rdiff) => {"ansible_loop_var": "item", "changed": true, "failed_when_result": true, "item": "11-system.rdiff"} failed: [debian-12] (item=20-db.mysql) => {"ansible_loop_var": "item", "changed": true, "failed_when_result": true, "item": "20-db.mysql"} failed: [debian-12] (item=21-db.rdiff) => {"ansible_loop_var": "item", "changed": true, "failed_when_result": true, "item": "21-db.rdiff"} PLAY RECAP ********************************************************************* debian-12 : ok=3 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 ``` I will see if using `block` is needed to group together some checks, or simply to always see all checks. What is weird, is that `package` in `check_mode` fails by default without using this trick. It’s inconsistent. ```auto TASK [Check backupninja installation] ****************************************** fatal: [debian-12]: FAILED! => {"changed": false, "msg": "No package matching 'backupninja' is available"} PLAY RECAP ********************************************************************* debian-12 : ok=1 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 ``` ---
**Author:** ![Thulium-Drake](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/thulium-drake/32/833_2.png) [@Thulium-Drake](https://forum.ansible.com/u/Thulium-Drake)\ **Post date:** [July 12, 2025, 1:53pm UTC](https://forum.ansible.com/t/molecule-make-tests-fail/43698/8 "2025-07-12T13:53:07Z")
That’s probably because some modules change their behavior when running in `check_mode`, but that’s not always what you want/need when running validation tests. The `yum_repository` role for example does need to make changes on the local test node in order to validate the function created by the role → providing a working YUM repo that can be used to install packages. So when that task can’t make changes, because `check_mode` doesn’t allow installing a package, the test is worthless 🙂 ---
**Author:** ![system](https://us1.discourse-cdn.com/ansible/original/1X/1882d9a01da5d8996a862359b0913cc568a39b84.svg) [@system](https://forum.ansible.com/u/system)\ **Post date:** [August 11, 2025, 1:53pm UTC](https://forum.ansible.com/t/molecule-make-tests-fail/43698/9 "2025-08-11T13:53:55Z")
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.