# Starting Openstack VMs in parallel **URL:** \ **Category:** Ansible Project\ **Tags:** ubuntu\ **Created:** [July 10, 2014, 10:17pm UTC](https://forum.ansible.com/t/starting-openstack-vms-in-parallel/15647 "2014-07-10T22:17:48Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![Kurt\_Yoder](https://avatars.discourse-cdn.com/v4/letter/k/da6949/32.png) [@Kurt\_Yoder](https://forum.ansible.com/u/Kurt_Yoder)\ **Post date:** [July 10, 2014, 10:17pm UTC](https://forum.ansible.com/t/starting-openstack-vms-in-parallel/15647/1 "2014-07-10T22:17:48Z")
Hello list, I anticipate provisioning 10-20 VMs using Ansible, then assigning floating IPs to each, then waiting for SSH to become available for each VM. I would like to do this in parallel instead of serially. Specifically: - Start the VMs, but don’t block - Assign the IPs, but don’t block - Wait on SSH until all VMs respond I saw the nova\_compute “wait: ‘no’” option, but when I use it I get a stack trace: > failed: [localhost] =\> (item=1) =\> {“failed”: true, “item”: 1, “parsed”: false} > invalid output was: Traceback (most recent call last): > File “/home/ubuntu/.ansible/tmp/ansible-tmp-1405028178.0-234314980043958/nova\_compute”, line 1490, in > main() > File “/home/ubuntu/.ansible/tmp/ansible-tmp-1405028178.0-234314980043958/nova\_compute”, line 266, in main > \_create\_server(module, nova) > File “/home/ubuntu/.ansible/tmp/ansible-tmp-1405028178.0-234314980043958/nova\_compute”, line 194, in \_create\_server > private = [x[‘addr’] for x in getattr(server, ‘addresses’).itervalues().next() if x[‘OS-EXT-IPS:type’] == ‘fixed’] > StopIteration Perhaps I’m using it incorrectly: > - name: Launch cluster VM on Openstack > nova\_compute: > name: “{{ os\_username }}\_cluster1” > state: present > login\_username: “{{ os\_username }}” > login\_tenant\_name: “{{ os\_tenant }}” > login\_password: “{{ os\_password }}” > image\_id: “{{ os\_image\_id }}” > key\_name: “{{ os\_username }}\_controller\_key” > wait: “no” > flavor\_id: “{{ os\_flavor\_id }}” > auth\_url: “{{ os\_url }}” > user\_data: “#cloud-config\nmanage\_etc\_hosts: true” So, two questions: 1. Am I using “wait” correctly? 2. Should I use “wait” to get to my desired parallel VM launch, as described above, or should I use something else, e.g. “async”? Thanks, -Kurt ---
**Author:** ![Kurt\_Yoder](https://avatars.discourse-cdn.com/v4/letter/k/da6949/32.png) [@Kurt\_Yoder](https://forum.ansible.com/u/Kurt_Yoder)\ **Post date:** [July 11, 2014, 9:20pm UTC](https://forum.ansible.com/t/starting-openstack-vms-in-parallel/15647/2 "2014-07-11T21:20:34Z")
Well, “async” is totally a bust. I got a message: ` fatal: [localhost] =\> lookup plugins (with\_\*) cannot be used with async tasks ` ---
**Author:** ![Kurt\_Yoder](https://avatars.discourse-cdn.com/v4/letter/k/da6949/32.png) [@Kurt\_Yoder](https://forum.ansible.com/u/Kurt_Yoder)\ **Post date:** [July 11, 2014, 9:28pm UTC](https://forum.ansible.com/t/starting-openstack-vms-in-parallel/15647/3 "2014-07-11T21:28:34Z")
I guess that error is because I put a “with\_items” in there. How does everyone else do this? I don’t understand how to loop asynchronously. See pseudo-code: \< start all 5 at once: \> \< start up openstack host \> \< assign it a floating ip \> \< capture the floating ip \> \< end when all 5 have a floating ip \> \< wait for all 5 floating IPs to have an open SSH port \> ---
**Author:** ![Michael\_DeHaan1](https://avatars.discourse-cdn.com/v4/letter/m/5e9695/32.png) [@Michael\_DeHaan1](https://forum.ansible.com/u/Michael_DeHaan1)\ **Post date:** [July 11, 2014, 10:34pm UTC](https://forum.ansible.com/t/starting-openstack-vms-in-parallel/15647/4 "2014-07-11T22:34:45Z")
So some of the provisioning modules, like AWS in particular, support spinning up “N” modules at a time by passing “count” or “exact\_count”. Rackspace I believe does this with manual looping (for now), but I could be wrong and that might have just been historical truth. If the OpenStack API says we can launch 10 at once, it could be made to do similar things. ---
**Author:** ![Kurt\_Yoder](https://avatars.discourse-cdn.com/v4/letter/k/da6949/32.png) [@Kurt\_Yoder](https://forum.ansible.com/u/Kurt_Yoder)\ **Post date:** [July 11, 2014, 10:54pm UTC](https://forum.ansible.com/t/starting-openstack-vms-in-parallel/15647/5 "2014-07-11T22:54:15Z")
So should I make a custom module which loops over nova\_compute asynchronously, and also assigns floating IPs? Would such a module be useful to the wider community, or is it too specialized to contribute back? ---
**Author:** ![Michael\_DeHaan1](https://avatars.discourse-cdn.com/v4/letter/m/5e9695/32.png) [@Michael\_DeHaan1](https://forum.ansible.com/u/Michael_DeHaan1)\ **Post date:** [July 12, 2014, 1:50am UTC](https://forum.ansible.com/t/starting-openstack-vms-in-parallel/15647/6 "2014-07-12T01:50:02Z")
“So should I make a custom module which loops over nova\_compute asynchronously, and also assigns floating IPs?” I’d first rather know where the openstack API allows simultaneous creation of N virtual machines of the same image type. I expect the floating IP stuff is fast and a usual with\_items loop isn’t a problem there, once those guests exist. (Using neutron, I assume?) ---
**Author:** ![Kurt\_Yoder](https://avatars.discourse-cdn.com/v4/letter/k/da6949/32.png) [@Kurt\_Yoder](https://forum.ansible.com/u/Kurt_Yoder)\ **Post date:** [July 14, 2014, 2:16pm UTC](https://forum.ansible.com/t/starting-openstack-vms-in-parallel/15647/7 "2014-07-14T14:16:13Z")
There’s an undocumented min\_count and max\_count option in python-novaclient ServerManager.create() which looks like what you want [1]. -Kurt [1] [https://github.com/openstack/python-novaclient/blob/cc7364067f995bd952149cd172cebf083fe4dc99/novaclient/v3/servers.py#L427](https://github.com/openstack/python-novaclient/blob/cc7364067f995bd952149cd172cebf083fe4dc99/novaclient/v3/servers.py#L427) ```auto ``` ---
**Author:** ![Kurt\_Yoder](https://avatars.discourse-cdn.com/v4/letter/k/da6949/32.png) [@Kurt\_Yoder](https://forum.ansible.com/u/Kurt_Yoder)\ **Post date:** [July 14, 2014, 5:07pm UTC](https://forum.ansible.com/t/starting-openstack-vms-in-parallel/15647/8 "2014-07-14T17:07:59Z")
To answer your other question about neutron: I use “quantum\_floating\_ip” to assign IPs on Openstack. It would be very useful if these API calls could also be parallelized. I imagine a lot more time would be spent on 100 serial API calls than doing them in parallel. -Kurt ---
**Author:** ![Michael\_DeHaan1](https://avatars.discourse-cdn.com/v4/letter/m/5e9695/32.png) [@Michael\_DeHaan1](https://forum.ansible.com/u/Michael_DeHaan1)\ **Post date:** [July 14, 2014, 7:09pm UTC](https://forum.ansible.com/t/starting-openstack-vms-in-parallel/15647/9 "2014-07-14T19:09:18Z")
Yep, I think we’d be open to that. ---
**Author:** ![Kurt\_Yoder](https://avatars.discourse-cdn.com/v4/letter/k/da6949/32.png) [@Kurt\_Yoder](https://forum.ansible.com/u/Kurt_Yoder)\ **Post date:** [July 14, 2014, 7:31pm UTC](https://forum.ansible.com/t/starting-openstack-vms-in-parallel/15647/10 "2014-07-14T19:31:53Z")
Is this a “pull requests welcome” situation? If so, I can try my best… 😃 ---
**Author:** ![Kurt\_Yoder](https://avatars.discourse-cdn.com/v4/letter/k/da6949/32.png) [@Kurt\_Yoder](https://forum.ansible.com/u/Kurt_Yoder)\ **Post date:** [July 14, 2014, 7:37pm UTC](https://forum.ansible.com/t/starting-openstack-vms-in-parallel/15647/11 "2014-07-14T19:37:17Z")
Opened two github issues: - [https://github.com/ansible/ansible/issues/8131](https://github.com/ansible/ansible/issues/8131) - [https://github.com/ansible/ansible/issues/8132](https://github.com/ansible/ansible/issues/8132) ---
**Author:** ![Michael\_DeHaan1](https://avatars.discourse-cdn.com/v4/letter/m/5e9695/32.png) [@Michael\_DeHaan1](https://forum.ansible.com/u/Michael_DeHaan1)\ **Post date:** [July 14, 2014, 7:57pm UTC](https://forum.ansible.com/t/starting-openstack-vms-in-parallel/15647/12 "2014-07-14T19:57:22Z")
Minor process note: If you’re going to submit the code to try, don’t open the feature requests, just clutters up the queue. If you aren’t, opening an RFE usually doesn’t mean the code fairies implement it anyway, since we’re likely going to attend to existing bug reports/PRs and feature PRs first ---
**Author:** ![Kurt\_Yoder](https://avatars.discourse-cdn.com/v4/letter/k/da6949/32.png) [@Kurt\_Yoder](https://forum.ansible.com/u/Kurt_Yoder)\ **Post date:** [July 15, 2014, 5:27pm UTC](https://forum.ansible.com/t/starting-openstack-vms-in-parallel/15647/13 "2014-07-15T17:27:29Z")
I dug a bit further. The API does allow min\_count and max\_count, much the way boto does for AWS. When you submit a request with min\_count, your instances are named -. That’s acceptable, though not ideal. I’m taking a look at the Ansible ec2 module, and the code for boto instance-launching code looks very different than the nova\_compute instance-launching code. I haven’t run it yet; I need to dig around to find my ec2 creds so I can run a test. The Ansible ec2 module also allows one to assign public IPs while launching multiple instances. The Ansible nova\_compute module does not permit this ATM. ---
**Author:** ![Michael\_DeHaan1](https://avatars.discourse-cdn.com/v4/letter/m/5e9695/32.png) [@Michael\_DeHaan1](https://forum.ansible.com/u/Michael_DeHaan1)\ **Post date:** [July 16, 2014, 9:02pm UTC](https://forum.ansible.com/t/starting-openstack-vms-in-parallel/15647/14 "2014-07-16T21:02:28Z")
Additions of new params to add IP spawning behavior would be reasonable. (assign\_public\_ip, True/False, etc) What might you prefer on names? ---
**Author:** ![Kurt\_Yoder](https://avatars.discourse-cdn.com/v4/letter/k/da6949/32.png) [@Kurt\_Yoder](https://forum.ansible.com/u/Kurt_Yoder)\ **Post date:** [July 17, 2014, 2:04pm UTC](https://forum.ansible.com/t/starting-openstack-vms-in-parallel/15647/15 "2014-07-17T14:04:13Z")
I found a different approach to this problem. First create a cluster configuration file. ` $ cat cluster.yml ---
**Author:** ![Kurt\_Yoder](https://avatars.discourse-cdn.com/v4/letter/k/da6949/32.png) [@Kurt\_Yoder](https://forum.ansible.com/u/Kurt_Yoder)\ **Post date:** [July 17, 2014, 2:19pm UTC](https://forum.ansible.com/t/starting-openstack-vms-in-parallel/15647/16 "2014-07-17T14:19:40Z")
I forgot to include a critical piece of this solution. Here’s how you invoke the task to tell Openstack to start the VMs: ` - name: Create cluster hosts: os\_api gather\_facts: no roles: [instantiate] # “instantiate” is a role that includes the Openstack startup task ` Perhaps this kind of example should be included in docs for parallel execution in Ansible. Let me know if you want me to help with that. -Kurt ---
**Author:** ![Tony\_Kinsley](https://avatars.discourse-cdn.com/v4/letter/t/b19c9b/32.png) [@Tony\_Kinsley](https://forum.ansible.com/u/Tony_Kinsley)\ **Post date:** [July 18, 2014, 2:33pm UTC](https://forum.ansible.com/t/starting-openstack-vms-in-parallel/15647/17 "2014-07-18T14:33:04Z")
So I currently have an inventory script that creates me “fake” devices which I run the nova\_compute task against. This allows me to provision all the vms in parallel using the native parallelization that ansible provides which is nice cause I can set it up to only provision 10 - 20 vms at a time using the serial flag. So this method provisions the VMs great, but since the “fake” devices do not have connection information I cannot use any other modules later in the playbook. I am trying to add these newly created VMs to a new group using the add\_host module but I cannot get it to add more than one device to the running inventory. - name: spawn vms local\_action: module: nova\_compute name: {{ vm\_name }} … register: nova - name: add new hosts local\_action: module: add\_hosts name: {{ nova.private\_ip }} I saw the fake device approach on another thread and liked the ability to use ansible to throttle my provisioning as well as choose the names for the vms. I was wondering though if it would be possible to update host with the connection information ( specifically the ansible\_ssh\_host, ansible\_ssh\_private\_key\_file, and whatever variable specifies how to connect to the host ) rather than creating new hosts. I also am stuck trying to get add\_hosts to get all of my hosts. I assumed that add\_hosts would also run in parallel but after searching around all of the examples I have seen show it using a loop. If I need to use a loop is there a good way to get the registered output from nova into a list rather than per host? I could not figure out any way to use set\_fact to append the per host values of the registered nova variable into a single variable. I felt like this approach was pretty clean, but I am definitely open to other ideas. My goal is to be able to provision a scalability test environment so I am expecting to be provisioning 500 VMs in a private Openstack network. Tony ---
**Author:** ![Michael\_DeHaan1](https://avatars.discourse-cdn.com/v4/letter/m/5e9695/32.png) [@Michael\_DeHaan1](https://forum.ansible.com/u/Michael_DeHaan1)\ **Post date:** [July 18, 2014, 6:19pm UTC](https://forum.ansible.com/t/starting-openstack-vms-in-parallel/15647/18 "2014-07-18T18:19:33Z")
I’m not sure why you think add\_host needs to run in parallel, it completes in almost zero time since it’s just doing some internal inventory “math”. That all being said, what we really want is OpenStack’s module having the equivalent of exact\_count in AWS/Rax other (though the Rackspace implementation is a bit more low level). ---
**Author:** ![Bob\_Tiernay](https://avatars.discourse-cdn.com/v4/letter/b/b3f665/32.png) [@Bob\_Tiernay](https://forum.ansible.com/u/Bob_Tiernay)\ **Post date:** [July 21, 2014, 12:03am UTC](https://forum.ansible.com/t/starting-openstack-vms-in-parallel/15647/19 "2014-07-21T00:03:05Z")
Hey Kurt, Do you happen to have a full example in a repo somewhere? I would be very interested to check it out. Thanks in advance, Bob ---
**Author:** ![Bob\_Tiernay](https://avatars.discourse-cdn.com/v4/letter/b/b3f665/32.png) [@Bob\_Tiernay](https://forum.ansible.com/u/Bob_Tiernay)\ **Post date:** [July 21, 2014, 1:02am UTC](https://forum.ansible.com/t/starting-openstack-vms-in-parallel/15647/20 "2014-07-21T01:02:55Z")
FYI - [http://docs.openstack.org/api/openstack-compute/2/content/POST\_multiple-create-v2\_createServer\_\_v2\_\_tenant\_id\_\_servers\_ext-os-multi-server-create.html](http://docs.openstack.org/api/openstack-compute/2/content/POST_multiple-create-v2_createServer __v2__ tenant_id__servers_ext-os-multi-server-create.html) To get exact\_count semantics, set min\_count and max\_count to the same value. This is part of the public OpenStack API. Cheers, Bob [Next page](https://forum.ansible.com/t/starting-openstack-vms-in-parallel/15647.md?page=2)