vmware/esx/vsphere module - planning

Archives Ansible Project
21

I started with vsphere_guest but had to give up for lack of the ability to clone from a vm. There was talk about it, but i don’t think it ever made it to trunk.

It’s unfortunate perhaps that vsphere_guest relies on pysphere whereas the current effort depends on pyvmomi which is vmware’s officially supported python API.

I need to clone from vm and supply disks. vsphere_guest has the disk support and ansible-vsphere has clone from template.

pyshere has a headstart in terms of exposed (documented) functionality and community, but will likely be eclipsed by pyvmomi if it can garner a community. Maybe for now a hybrid approach is necessary with pip-pointed modules dependent on both. It’s no big deal for for a vsphere lover to install both dependencies, although likely one will win in the end.

22

It would be much better to add that capability to the existing one than bifurcate the modules.

See this pull request, though it hasn’t had comments answered on it - https://github.com/ansible/ansible/pull/7690

Though I would tend to think maybe snapshot is a different module, and cloning is a parameter of the existing one.

Send me a pull request for an update and we can definitely prioritize it, I don’t want to see these start to fork - there’s no good reason for that.

23

Well it’s unfortunate that their is so duplicated functionality. I actually started this before that module was part of core, and when it finally did get added to core it did not meet my needs. That said my module does a lot more than the one in core. Like I said before you can manage VM snapshots, and interact with the guest operations manager which I use to reconfigure the networking on the devices after I clone the VMs. My module can also power the VMs on and off.

Unfortunately my module does not provide good output after the tasks, and I really could use some help extending it.

As for attaching volumes I have never done that but I think you should be able too. Are you talking about attaching them during a Clone or Create process or attaching a volume to an existing VM?

I think something like this should work. If you look at the spec values they match up to the vsphere API identically. The spec type is the name of a function to run on the guest managed object and the value of the spec is the parameters to that function ( which in this case happens to be called spec ). You do need to fill in any required fields for those objects, which are specified in the vmware documenation.

`
-name:attach volume
local_action:
module:vsphere
guest:
name:test
state: running
action: task
spec:
type: ReconfigVM_Task
value:
spec:
VirtualMachineConfigSpec :
deviceChange:

  • VirtualDeviceConfigSpec:
    operation: add
    device:
    VirtualDisk:
    diskObjectId:

`

24

"hat said my module does a lot more than the one in core. "

Right, so I’m suggesting let’s add functionality to the core one :slight_smile:

Please send me a pull request if this is something you think you’d like to do - it’s way better long term, and we’ve always been about “batteries included” and getting everybody together to make modules that work for everyone.

We do need to keep the existing signature for module parameters it currently takes, and I do want to have the conversation about the form factor of extensions to it.

25

After using both, I definitely like using pyvmomi better. Since the API is almost identical to the Vsphere API I was able to use not only the API docs from vmware, but docs for rbvmomi, rvc, and even the java vmware sdk and apply it towards getting the module to do what I want.

As for adding a drive to the cloned VM, that’s definitely doable. Here is my example ( that I actually use ) that shows adding a network device.

`

  • name: clone vsphere machine
    local_action:
    module: vsphere
    host: “{{ vcenter_host }}”
    login: “{{ vcenter_login }}”
    password: “{{ vcenter_pwd }}”
    timeout: 60
    guest:
    name: “{{ host }}”
    state: present
    folder: “{{ group }}”
    clone_from: “{{ template }}”
    spec:
    type: VirtualMachineCloneSpec
    value:
    config:
    VirtualMachineConfigSpec:
    name: “{{ schema.hostname }}”
    memoryMB: 4096
    numCPUs: 1
    deviceChange:
  • VirtualDeviceConfigSpec:
    operation: add
    device:
    VirtualVmxnet3 :
    key: 0
    backing:
    VirtualEthernetCardNetworkBackingInfo:
    deviceName: “{{ networks[0].name }}”
    location:
    VirtualMachineRelocateSpec:
    pool:
    ManagedObjectReference:
    type: ResourcePool
    name: ‘{{ resource_pool }}’
    host:
    ManagedObjectReference:
    type: HostSystem
    name: ‘{{ compute_host }}’
    datastore:
    ManagedObjectReference:
    type: Datastore
    name: ‘{{ datastore }}’
    powerOn: True
    template: False

`

To add a disk you would just need to change the deviceChange list to include a VirtualDeviceConfigSpec for the hard drive that you want. I know this is not nearly as easy as just specifying a hard drive, but by taking this approach you can now get any feature that the vmware api provides such as the type of disk backing you want.

26

I’m open to anything that keeps the same compatible Ansible module signature.

27

One thing that is missing from the current vsphere module is the ability reconfigure the cdrom device. My current build process uses a boot.iso to start the build (dont ask why) and so at the end of the create I want to remove the cdrom.

I’m currently using an external script that calls pysphere to do this, it would be nice to have this built in to the ansible module - I’m struggling to work out how to do this. Tony would your module handle this?

28

Snapshots let you store the state of a vm to roll back to later. It’s different from cloning. I searched the module for “clone” and “template” and neither is there.

The examples don’t cover all the config options, nor do the docs.

vm_hardware:

memory_mb: 2048

num_cpus: 2

osid: centos64Guest

scsi: paravirtual
but when I played with vsphere_guest my conclusion was that the only support for selecting the base for the vm was def add_cdrom(module, s, config_target, config, devices, default_devs, type="client", vm_cd_iso_path=None):

In vmware, there are two very different paths for creating a vm. One, you select the cd_rom which is like a bootable iso. The other (more common) is to build up a template (i believe it is backed by a vmdk) and then clone from the template. The latter is the functionality i need, and the former is all that is supported. I could be wrong, but would need an example to show cloning with the given api.

Tony, I am talking about adding a volume after boot, for logs or data drives, although creating partitions on boot would be nice too.

Vsphere is capable of sooo much, I think the idea of adding hooks that expose the functionality in an ‘extra-params’ like way is the way to go, otherwise you end up with a dozen vsphere-x modules. If vsphere_guest has that capability and it just needs exposing, great. If it is fundamentally not extend then perhaps other options should be explored.

I think apt and synchronize already offer this type of “deeper through extra-args” functionality. Perhaps the best option would be a master module capable of pretty much everything (with good documentation) and then a few facades modules for the most common usages. The code should be factored out into a handful of modules plus some helper classes that are used by all modules, like authentication and connection.

The trouble with the current vsphere_guest is it offers insufficient capabilities to satisfy any real-world usage pattern, and there doesn’t seem to be much activity in extending it to do so. Tony’s module with the built in power via spec can do orders of magnitude more but would need lots of examples for the various use cases.

I will try to play with the spec today to see if it meets my needs.

29

I’m fairly confident my module can do it, I just have never done that with the vsphere API. If you have already written a script to do this in pysphere it should be pretty simple to port that. I am assuming that you want to do a ReconfigVM_Task and it would look something like this below. One thing my module is definitely missing is a good facts gathering system. So you would need to gather the device key information in your inventory script.

`

local_action:
module: vsphere
host: “{{ vcenter_host }}”
login: “{{ vcenter_login }}”
password: “{{ vcenter_password }}”
timeout: 60
guest:
name: “{{ deleteme }}”
state: running
action: task
spec:
type: ReconfigVM_Task
value:
spec:
VirtualMachineConfigSpec:
deviceChange:

  • VirtualDeviceConfigSpec:
    operation: remove
    device:
    VirtualCdrom:
    key:

`

30

“I think the idea of adding hooks that expose the functionality in an ‘extra-params’ like way is the way to go”
It’s not.

We really want to model things as nouns, so the vsphere_guest module should take a parameter that can boot from a cloned image.

31

Hi all,
I’ve added quite a bit to the vsphere_guest module to fit our needs, but haven’t submitted PR due to lack of time to generalize / create documentation. I’m also the author of PR 7609 referenced above. Things that I’ve added are the ability to change dvs switch as part of reconfigure, clone, and expand disk as part of reconfigure_vm. If anyone is interested is taking a look at my crude implementation, I can put it up on github. Like I said, I haven’t put in PR yet becuase, for example, reconfigure_vm only reconfigures dvs switches, and not standard. and the other bits aren’t tested/documented well enough for me to be comfortable submitting. The green button is scary.

I’ll open a new thread regarding PR 7690

32

It seems a few of us have different implementations of cloning a template, or VM. here’s mine: Mine sets state to ‘clone’ (which probably should be ‘cloned’ or something more descriptive?) and then adds a parameter: template: to specify a template/VM in which to clone from. I had also noticed there is another PR out there to deploy from template.

https://github.com/whereismyjetpack/ansible/tree/clone

If you look at the branch you’ll see there is a lot of logic regarding folders, our organization uses VM folders heavily to grant access and divide systems, so I baked that in, too (it’s not documented yet)

I’m waiting on feedback from this PR to document the clone/new vm folders
https://github.com/ansible/ansible/pull/7792

P.S I’m also LogLevel9 – i somehow got two github accounts and I’m trying to merge them into one.

33

Hey guys, I’ve recently been looking to control vsphere via Ansible which lead me to this thread. Noticed a couple pull requests that were before the September 26th git core-module/extra-module split by the Ansible team. Just wanted to revive this thread for those that had PRs that were closed before resolution.

Cheers,

34

Hi Tony,

I was playing with vsphere module which you had written.

Vmware tools option am trying

35

Hey Nitin,

That error looks really strange, I do not know where “Status API Training Shop Blog About” comes from. But syntax errors are usually pretty easy to figure out.

Try export ANSIBLE_KEEP_REMOTE_FILES=1 and rerun the playbook verbose again. After it fails it will not clean up the module that it copies over to the host that gets executed on, so you should be able to navigate to the directory it stored it in and view the module itself. Then you can check out what line of code corresponds to line 2491.

Also what version of ansible are you using? Not sure how well you know the inner workings of ansible, but basically it takes the custom modules and injects them into a much larger python file that contains loads of helper functions and classes that can be used by the custom modules. I think in newer versions ansible injects the custom module at the top of the file so when you see a backtrace and a line number it matches up with the module itself. Seeing the issue at line number 2491 also makes me think there might be something else going on.

Let me know how it goes.
Tony

36

Hi Tony,

Ansible version am using is 1.8.2. I checked line of code corresponding to line 2491. It says "Status API Training Shop Blog About". No other clue:(.

Last few lines are as below,

def pretty_bytes(self,size):
ranges = (
(1<<70L, ‘ZB’),
(1<<60L, ‘EB’),
(1<<50L, ‘PB’),
(1<<40L, ‘TB’),
(1<<30L, ‘GB’),
(1<<20L, ‘MB’),
(1<<10L, ‘KB’),
(1, ‘Bytes’)
)
for limit, suffix in ranges:
if size >= limit:
break
return ‘%.2f %s’ % (float(size)/ limit, suffix)

def get_module_path():
return os.path.dirname(os.path.realpath(file))
main()
Status API Training Shop Blog About
© 2015 GitHub, Inc. Terms Privacy Security Contact

and verbose output is as below,

PLAY [127.0.0.1] **************************************************************

TASK: [Upgrade Tools] *********************************************************
<127.0.0.1> REMOTE_MODULE vsphere host=xxx password=VALUE_HIDDEN login=administrator@vsphere.local
<127.0.0.1> EXEC [‘/bin/sh’, ‘-c’, ‘mkdir -p $HOME/.ansible/tmp/ansible-tmp-1424886684.83-207068257700957 && chmod a+rx $HOME/.ansible/tmp/ansible-tmp-1424886684.83-207068257700957 && echo $HOME/.ansible/tmp/ansible-tmp-1424886684.83-207068257700957’]
<127.0.0.1> PUT /tmp/tmp3gLdMW TO /home/osboxes/.ansible/tmp/ansible-tmp-1424886684.83-207068257700957/vsphere
<127.0.0.1> EXEC [‘/bin/sh’, ‘-c’, u’LANG=C LC_CTYPE=C /usr/bin/env python /home/osboxes/.ansible/tmp/ansible-tmp-1424886684.83-207068257700957/vsphere; rm -rf /home/osboxes/.ansible/tmp/ansible-tmp-1424886684.83-207068257700957/ >/dev/null 2>&1’]
failed: [127.0.0.1 → 127.0.0.1] => {“failed”: true, “parsed”: false}
File “/home/osboxes/.ansible/tmp/ansible-tmp-1424886684.83-207068257700957/vsphere”, line 2491
Status API Training Shop Blog About
^
SyntaxError: invalid syntax