I have started creating some roles for my use and implemented some tests using Molecule which now uses Ansible as verifier.
I’m not sure where I saw it but I have implemented tests using standard Ansible modules but with check_mode: true which is fine but doesn’t makes the verify step fail, errors are reported as changes.
I think there might be some misunderstanding of the verify stage. The verify job should check whatever values/settings/state you need and then explicitly fail the job if they aren’t the way you expect.
Issue #1 - The lininfile module isn’t going to fail it in the way that you want, regardless of how you set check mode. You could use it to register a variable and then assert the value of that variable later to determine if the job should fail.
Issue #2 - Molecule should ideally be permitted to run destructive tests since it provides the capability to create and destroy test systems. Its not clear to me if you’re using check mode to “protect” the test system. If that is the case, I would suggest against it and instead provision tests systems using the create stage.
I created an example molecule setup for use in the Red Hat Developer Sandbox. It is a simple role to make a backup of a file and then verify multiple aspects of that operation. You’ll notice that it doesn’t fail the job until the very end so that I can gather up all the failures rather than bailing on the first issue. Example code can be found here.
If you want to test this out yourself, there is a blog here that shows you how to spin up the free environment to experiment with this specific example.
I know that because it runs in a temporary container or VM, whatever happens in Molecule has no impact. (I previously used Test-kitchen with Salt and Inspec on Vagrant VM.)
I used check_mode because I think I saw it somewhere in an article or molecule tests.
I thought it could be used for that.
If I understand your example correctly, you use ansible.builtin.assert to test your molecule setup by verifying the variables you ‘register’ using other methods (stats or whatever).
Does that mean that in my case I could continue using my lineinfile or template or other module I currently use by simply registering the result and then testing it with assert.
For me it looks like writing the tests will be longer, with 2 or 3 steps instead of only one (when I used Inspec)
For me it looks like writing the tests will be longer, with 2 or 3 steps instead of only one (when I used Inspec)
what I mean by saying that:
instead of
whatever_method_to_verify
do I need to write
one_verification:
register: variable
assert:
variable is defined and variable is equal to "xx"
?
It seems quite verbose to me and I would like to know if there is any way to test/verify in one step. But maybe it’s not possible using Ansible as a verifier.
Using failed_when and ansible.builtin.assert is basically most you need. You can use it to check values or compare stuff. You could even have (though molecule should already do that for you in the idempotency test) have it fail like this:
Yes, that is often the way. There are some tests that may not require variables being registered. For example if your role installed a service, you may include a test that just starts the service. In that case you may not need the extra granularity that comes with the register/assert.
Your example didn’t include the block rescue, but I’d also encourage the use of that. That way you can capture multiple failures in a single test. It also allows you to include negative/expected failure tests.
That’s probably because some modules change their behavior when running in check_mode, but that’s not always what you want/need when running validation tests.
The yum_repository role for example does need to make changes on the local test node in order to validate the function created by the role → providing a working YUM repo that can be used to install packages. So when that task can’t make changes, because check_mode doesn’t allow installing a package, the test is worthless
