Contributing new plugin, adding libcloud as a dependency

Archives Ansible Project
1

Hi,

I’m considering contributing a new plugin to the Ansible community. Ideally, I’d like to use libcloud [http://libcloud.apache.org] but I believe this would introduce a new dependency so I wanted to check first. It may even be possible to standardize some of the plugins on libcloud (e.g. RAX, EC2, Linode, etc).

Any objections to at least introducing libcloud for the new plugin?

Cheers,
Eric

2

What is the plugin for if I may ask?

I am fine with modules having deps as long as they check for them and report if they aren’t available in clean ways.

I generally would /guess/ the native bindings might provide more options in many cases (boto, pyrax, etc), but I’d yield to the module authors in that regard.

3

Yes, I certainly wouldn’t consider moving away the AWS modules from boto in replacement of libcloud. It’s got a long long way to go.

4

I’ve started messing around on a plugin for Google Compute Engine.

I have two likely options:

  1. libcloud which was just recently updated to support GCE. Under the hood, it uses paramiko and pycrypto so I don’t think you’d see a lot of extra deps for ansible plugin users if this approach were taken.
  2. Use google-api-python-client and oauth2client api but that introduces two top-level dependencies along with their own set of deps. This approach would have broader feature coverage than using libcloud (to Lester’s point).

I’m leaning toward libcloud at this point and it sounds like there are no real concerns as long as I do the proper checking.

5

I would go for (2). There will absolutely be features people want which libcloud doesn’t support.

6

libcloud adheres to a minimal “contract” that it tries to maintain across all providers for each core service through a defined API and set of classes. It encourages additional features outside of that contract through “ex_” methods. It also provides a lot of boilerplate functionality (retries, connection pools, etc) that are lacking with option 2.

7

Yeah a least common denominator solution would not be good.

I’d be very inclined to depend on the google client.

8

I believe I’m doing a poor job arguing the case for using libcloud for adding GCE support to ansible. Please note that I’m only arguing the case for using libcloud to add GCE support; standardizing ansible on libcloud is not my concern.

I truly believe the best way to accomplish adding GCE support in ansible is to use libcloud versus google-python-api-client. Please bear with me while I explain in more detail. Below is a feature comparison that I believe shows that libcloud more than covers all the use-cases an ansible user would care about. Furthermore, I’d make the case that libcloud provides more functionality in the sense that it has authentication/authorization, SSH support, blocking/non-blocking calls, and convenience functions for creating/destroying multiple instances.

Feature libcloud google-api-python-client comments

9

Perhaps using libcloud for this one is ok if it does what we need it to.

I do not want to see other modules switched to libcloud if it means we are not able to get features as quickly or we have to remove anything, i…e I don’t see us generically just using libcloud.

10

Thanks Michael. I do want to provide a quality full-featured ansible plugin for GCE. Ideally, I’ll have something to show within the month.

11

Ah, see that GCE example looks quite attractive. Question though: where the heck are the current libcloud docs? Even http://libcloud.apache.org/apidocs/0.13.0/moduleIndex.html doesn’t mention the GCE driver. Are Google working upstream in libcloud?

12

Hmmm… Not sure how/when those docs are generated.

I’ve been using the source for the gce driver along with the demo script as “documentation”. When my colleague that contributed the work returns from vacation I’ll ask him to follow up with the upstream team about getting their api docs updated. The GCE work is a very recent addition having just been accepted on 12-Jul.