Ansible approved playbook testing?

Archives Ansible Project
1

Dear Ansible (Michael deHaan?),
Hello! I’ve searched the google & the mailing list for playbook testing strategies and haven’t found anything that looks a consensus on how this should be done. Perhaps there are some projects, e.g. ansible-spec or ansible-cucumber, in the pipeline, but until then, is there a “best practice” for ansible testing?

At the NYC ansible-fest the other day there was a talk about testing, but the method discussed involved using servermaint, which is ruby and seems to irritate python people . . . the moderator of the fest (Adm. deHaan) said to instead use the scripts & fail modules - at least I think that’s what was said - but I haven’t been able to find anything about them pertaining to testing. Does anyone have a doc about this? An example?

Thanks a lot!!!
Guy

2

I was also at Ansiblefest, and I believe someone mentioned the assert module as a method of testing.
Personally, I wasn’t put off by using serverspec and I’m going to give that a try.

3

In the checkout under test/integration you can find playbooks with the assert module, these are used in ansible development to make sure changes don’t break expected functionality.

It is just as easy to create them to verify server status.

4

The point was that a lot of people think server_spec is somehow required to test “behind” Ansible to see what it did what it said it did.

It’s not.

The reason is because this is the whole purpose of a configuration management system - to say “the state shall be X” and make it so.

Thus what you really want to do is run integration tests.

In Ansible, especially in doing a rolling update, calling a test script prior to your “post_tasks” section where you add something back to a load balancer is a GREAT way to make sure failed nodes don’t hop back in. (set max_failed_percentage: X first to choose how many nodes can fail before the update is stopped).

assert, wait_for, script, and stat all make good modules to drive some basic testing.

I definitely would disagree that anyone would have to look at infrastructure like code, that’s the whole point of Ansible is saying this is over complication.

Ansible playbooks aren’t code, they are more lists of things that should be done, or true, in order.

The best way of testing is often just to do a full playbook run against a stage environment, and then run your test battery against that stage environment.
That is often going to be something a lot more involved that your QA team might write - check to make sure the app actually works, rather than what server_spec does, which is usually (in all examples I’ve seen) usually weak stuff like “is this port open”. Which doesn’t prove your app actually works – and has the added overhead of (A) being a different language (B) being a duplication of effort, etc.

I owe a blog post on this, which I hope to get done this week. TBD.

So, that’s not saying no to testing, but it’s saying no to writing tests that are a mirror of the low level descriptions of your infrastructure that you already wrote playbooks for.

Integration testing is where it’s at, and having ansible playbooks call those – especially in a rolling update – is a GREAT idea, since ansible is a fail-fast system and it will show up in your server report at the bottom.

There are some systems that will keep configuring as much as you can, but Ansible’s going to raise it to your attention and make you fix it :slight_smile: