Hi
To fast forward running integratino tests on outstanding PRs, I created
a simple integration test setup to run tests on any PR by giving a few
parameters.
I started to run integration tests on Debian related distros (Debian7+8,
Ubuntu12-16.04). Later I'll extend the test runs to CentOS 6+7.
The test setup playbook provisions the VMs as workers (one per distro)
on the Exoscale public cloud running a playbook on jenkins and run the
tests on these workers. A complete test run on all distributions in
parallel takes about 16 min.
The results and logs can be viewed on
https://build.ngine.ch/view/ansible.test.pipeline/
The setup playbook can be found here
https://github.com/resmo/test-ansible and could easily adapted to any
other cloud (digital ocean, ec2). It creates the VM and installs the
dependencies on these VMs and finally creating scripts on jenkins, one
per worker, from a template. These scripts are executed as separate jobs
to get proper logs out of the test run.
Currently only tests on ubuntu 12.04 and 14.04 passes. An issue on
Debian 7 related to git module has been identified.
I am working on making the test run on Debian 8 and Ubuntu 16.04 and the
current WIP can be found here
https://github.com/ansible/ansible/pull/15561. Any help to make the test
pass is much appreciated.
I am not sure yet how I will run the tests in the future. I plan to run
them automated in some kind, but I don't see running them on every PR
automated, but I have some other ideas. For now I'll run them once per
day on devel branch.
If you like me to run test on a PR, just let me know by make a comment
e.g. "@resmo please run tests", for a specific target to run, e.g.
"@resmo please run tests, target non_destructive
Any help is welcome.
Regards
René