Best practices for using Ansible with and without Vagrant

Archives Ansible Project
1

First a little background …

I’m building my staging and production provisioning using Ansible and organized as suggested in the Ansible best practices section of the documentation. My Ansible playbooks are kept in an SCM (Git) repository called infrastructure.

I recently started using Vagrant on my development workstation to allow me to easily create or recreate a clean environment (as a guest VM) on my workstation. I’m storing the required Vagrantfile in SCM (Git) along with my project’s files and I want to use Ansible to also provision the Vagrant environment. Since much of the configuration of these guest VMs matches the production environment (that’s the whole point right?), I want to use the roles (and associated tasks) that are used to provision the production and staging servers when provisioning the VMs.

So the problem I have is …

The bulk of my provisioning is organized as recommended in the Infrastructure project’s repository, but I need access to those files from each of the projects that spins up guest VM(s) on my development system. Obviously I can check out the infrastructure project and refer to it from the other project (if I’m careful to check the projects out in the same directories), but that’s not at all portable. Another alternative is to check out the Infrastructure project as one of the tasks before the Vagrant specific provisioning happens, but that may lead to having multiple copies of the same repository on my HD (not a space problem) and I’m afraid I’d miss updating all of them if the infrastructure project has changes.

Is anyone else using Ansible for both their production provisioning and with Vagrant? What are the best strategies for coping with playbooks/plays/tasks/etc that reside in different Git (or other SCM) repositories?

Thanks,

Steve

2

You can easily use the same ansible tree with seperate inventory files, just pass in the “-i” flag in each case.

3

Yeah … and the inventory file for any of my projects is pretty short.

The real question is (perhaps) more related to managing two Git projects together. I want a convenient way of checking out my infrastructure project into the same workspace as my development project.

Steve

4

“I want a convenient way of checking out my infrastructure project into the same workspace as my development project.”

I don’t know what a workspace is.

5

This sounds more like an SCM issue than an Ansible issue, no?

Some thoughts:

  1. Combine the two projects into one. TBH - not sure why you have them separate, it makes this sort of thing far harder, as you can see.
  2. Git Submodules (http://git-scm.com/book/en/Git-Tools-Submodules). They can be a little tricky, but they really are designed for this sort of issue.

Mark

6

+1 Mark’s advice. If your project is laid out appropriately submodules would be a very good fit.
Ali

7

Hi All,

8

What Mark said.

I have a setup according to http://ansible.cc/docs/bestpractices.html

I have 2 hosts files in this directory:

# file: hosts
[webservers]
web-1.example.com
web-2.example.com

[monitoring]
mon-1.example.com

# file: vmhosts
[webservers]
webserver.vm

[monitoring]
monitoring.vm

In my Vagrantfile I have a vm for each group:

# file: Vagrantfile

Vagrant::Config.run do |config|

  config.vm.define :test do |vm_config|
    vm_config.vm.box = "precise32"
    vm_config.vm.host_name = "webserver.vm"
    vm_config.vm.network :hostonly, "192.168.100.10"

    vm_config.vm.provision :ansible do |ansible|
      ansible.playbook = "site.yml"
      ansible.hosts = "webservers"
      ansible.options = "-vvv", "--inventory-file=vmhosts"
    end
  end

end

You can optimize this by creating multiple Vagrantfile.[ansible_group] files if you have a lot of groups and don’t want to create 100 vm’s every time.

– Frans

9

Just in case you guys weren’t aware. Last week(?) I contributed an external inventory script for Vagrant:
https://github.com/ansible/ansible/blob/devel/plugins/inventory/vagrant.py

Handy if you don’t want to set a static ip for your VM, and you can just have ansible look up what IP and port your vms are on.

Since we were on the topic, thought I would point it out.

Mark

10

Cool! I wonder if we could get something like this accepted upstream into the vagrant project (although they’d probably prefer it in ruby…), so you could do something like:

config.vm.provision :ansible do |ansible|
ansible.playbook = “./provision/your_playbook.yml”
ansible.use_vagrant_inventory = true
ansible.verbose = true
end

11

That’s a great idea!

I may have a dig around in the source, see how easy that would be to add.

Mark

12

From this thread I notice that there is at the moment two “main” ways for Ansible+Vagrant users, which is quite confusing:

  • original Ansible Plugin: https://github.com/dsander/vagrant-ansible (or related forks).
  • Ansible provisioner is also part of Vagrant “core” since 1.2: https://github.com/mitchellh/vagrant/pull/1465. But this provisioner does not support (yet) parameters like ansible.options or ansible.hosts
    Since Vagrant now integrates Ansible in its own production line, I am more in favor to develop this second (new) way (but I have no experience with the plugin, so I am not able to compare). Do you also see that both usages should soon or later converge? I guess that maybe the Ansible Plugin has more features/activity, and thus prevent to make the shift in a convenient way…

About the ansible.use_vagrant_inventory idea: I also aim at in DRYing at maximum the collaboration between Vagrant and Ansible (at least for playbook development tasks). In Vagrant-1.2 the ansible provisioner, ansible-playbook is called for each VM to provision (I guess it is the same with dsander-like vagrant plugin, but I haven’t verified yet). In multi-machine mode, that leads vagrant to run ansible-playbook several times against the whole inventory (workflow mismatch between “vagrant vm looping” and “ansible parallelism”). I thus opened https://github.com/mitchellh/vagrant/pull/1723, which meets my needs by generating on the fly a single-host inventory file for each VM to provision. It is quite handy for playbook development (so far, I don’t plan to use vagrant for staging/prod site deployments). Of course this approach does not support [host groups].

If you have time and interest on improving Vagrant default ansible provisioner, I invite you to comment in https://groups.google.com/d/topic/ansible-project/qCa7Egg2Gvk/discussion and related pull requests. I’d happy to get some more confirmation/refutation on these points… and so ensure that ansible+vagrant “standard” toolchain converges soon, making ansible community less confused about these variants.

Gilles

13

If you are using Vagrant, you should use the new provisioner.

14

For me it’s also a problem that the Vagrant Ansible plugin calls the provisioner separately for each VM. I’d also like it to be run once after all VMs are up, to take advantage of Ansible’s parallelism. I opened an issue about this in Vagrant’s tracker:

https://github.com/mitchellh/vagrant/issues/1784

However, I don’t think this issue should be mixed with the auto-generation of an inventory file by Vagrant (which is a separate feature request). For example, I use the inventory file to pass some variables, so an auto-generated one is not an option for me.

15

Yeah I don’t know, is there a vagrant list to coordinate that on instead?

This is really not a part of Ansible.

I understand many people like it, but I personally like to use virtual machine snapshots instead for testing, and avoid the extra layers of abstraction that (to me) make things require more hoops to jump through.

You can then just revert those VMs back to normal if you want to do a new clean test.

Tim Gerla was doing some very nice things with “vmrun” exactly for this purpose.

16

FYI - if anyone can think of a way that the vagrant external inventory plugin can better handle multi-machines in vagrant, I’m all ears. Config file maybe?

Mark

17

ansible.hosts: is ansible.limit now in vagrant 1.2 (see http://docs.vagrantup.com/v2/provisioning/ansible.html #ADDITIONAL OPTIONS )

ansible.options: there is a pull request (https://github.com/mitchellh/vagrant/pull/1697) which addsansible.raw_arguments
this same pull request also adds ansible.tags and a better ansible.verbose

– Frans

18

I’m totaly agree with you.

19

Nice script, I hadn’t seen that until now, but will likely start using it for a few of my setups.

Just wanted to chime in and say that for 99% of the infrastructure I manage, I just have on Git repo for both dev and production (and a few that have config for dev + test + production). I usually use separate inventory files for each environment, and for environment-specific variables, I do one of two things:

  1. Store the environment config in a separate application-specific repo and use a tool like Jenkins, Ant, Phing, etc. to deploy that vars file into the proper directory, prior to running the ansible playbook.
  2. Use host_vars and/or group_vars to separate out configuration within the main ansible repository.

Either method seems reasonable, depending on how you and your team deploys servers and apps, but I like the separated approach a little better, because the particular configurations are controlled by the application, whereas role defaults are usually sane defaults that environments can fall back to if they don’t care about specific configurations.

20

For this particular problem (calling on every vm) i use a trick i found online: only include the ansible provision call in the last vm in the Vagrantfile in is provider section.