I'm experimenting with ansible and wanted to know if there is an easy way to write test cases for each configurable task.
One of the philosophies of an older config mgmt system I wrote many years ago was to encourage authors to create automated test(s) for each change made to a system.
So, for example, if I have a module with a set of tasks to get sendmail working, we could run the module with a 'test' argument on client machines and have it report whether sendmail is configured correctly or not. I wouldn't care about each task within that module, just whether the entire goal of that module was accomplished. Scripts built on top of these could optionally skip those modules if the test cases succeeded. Audit reports across machines could summarize status in terms that were relatively easy to understand (compared to low level dry-run type of info).
Naturally, the quality of test cases has a bearing on effectiveness, and I realize that there's some duplication of effort involved -- code to effect changes, and code to test.
However, it is useful, and I'm curious how to accomplish the same with ansible?
If it's not possible today, I'd like to request this as a feature.
I'm having a bit of trouble with Google groups and their javascript-gone-wild, so I'll have to dig through the archives and read your thoughts on --check, which I happened to notice in passing a couple of days ago.
I use a couple of handlers as testers, nginx and nagios for example
can be executed with a 'sanity check' of their config files. I'm
looking into something for sudoers also.
Do you know how you would like to use such a feature?
I am not sure where it would belong in a playbook.
It sounds like it should be a per task parameter.
If we can decide on a good example usage I
can try to implement it, and see if anyone
likes it.
Just took a look at that thread. I'm not entirely sure it's what I'm
looking for, but I'm glad others are interested in something of this
nature. I'd be happy with playbook level tests, and the task level tests
would be a bonus.
Being fairly new to ansible and not well versed in what it can do, I was
merely stating the high level requirements.
Within my organization, I'm looking at alternatives to the existing tool
chain, which encourages:
1. Writing a unit test for every module that applies changes to a system
(in this case I'd say a module translates to playbooks consisting of
individual tasks)
2. Tight coupling of unit tests with the actual code that makes changes
To illustrate this, here's a pseudo example of steps to setup sendmail:
- stop sendmail if running
- copy conf file for given profile
- set correct perms on config file
- run chkconfig to enable sendmail service
- restart sendmail
I'd like to be able to say something like:
# ansible-playbook --brief --test sendmail.yml
[host1] sendmail.yml: FAILED
[host2] sendmail.yml: OK
# ansible-playbook --brief --run --failedonly sendmail.yml
[host1] sendmail.yml: OK
and unless I wanted more verbose output, I wouldn't care about the
individual tasks or task-level unit tests as much as that the macro changes
succeeded or failed.
Again, the above is simply to illustrate the point. I'll let more ansible-savvy folks translate ...