So, I’ve been thinking about creating a general purpose http client URI/URL module for ansible; allowing ansible to interact with web services. Some example use cases I’ve thought of:
Create a JIRA ticket on some condition using JIRA’s RESTful API
Delete/Update said JIRA ticket
Add a host to ‘your-favorite’ monitoring/CMDB system via it’s web API. (I use zabbix which has such an API.)
Delete said host from the monitoring/CMDB system.
Sending ansible ‘setup’ module json output to some yet created webservice for host-inventory cataloging.
Generally, just allowing ansible to interact with web-services.
Still in the planning stages but the biggest problem I see right now is that the standard & included HTTP client libraries in python have some rather big holes.
urllib2 only supports GET and POST HTTP methods.
httplib doesn’t appear to support authentication, proxies, compression, cookies or redirects.
That leaves either non-standard python http client libraries or curl. Sticking with python will certainly make the job easier. The two popular python http client libs I see are cover all/most features are:
httplib2 (big bonus in that it runs on python 2.4)
requests (doesn’t support python 2.4)
I’m tempted to use httplib2 for this since it’ll be sticking with python but also support python 2.4. But before I start working on this though I’d like to get feedback from others to see if I might be missing something glaringly obvious. While I realize this likely won’t be support in core due to external deps (?) I’d like to make this module useful to others as well.
BTW, for POSTS and PUTS, do you see that going from a template in many
cases? with_template could be very good for that.
XML should not /usually/ care about whitespace.
I also want to make AnsibleModuleCommon transparently start using JSON
which will cleanup some parsing dependencies anyway... and enable what
some people have been wanting about sending complex arguments to
modules.
BTW, for POSTS and PUTS, do you see that going from a template in many
cases? with_template could be very good for that.
Yes, I imagined templates would be quite useful in generating the
appropriate payload to send. Wasn't aware of 'with_template', thanks for
pointing that out.
XML should not /usually/ care about whitespace.
I also want to make AnsibleModuleCommon transparently start using JSON
which will cleanup some parsing dependencies anyway... and enable what
some people have been wanting about sending complex arguments to
modules.
So, I've been thinking about creating a general purpose http client URI/URL
module for ansible; allowing ansible to interact with web services.
Yes, please!
While I realize this likely won't be support in core due
to external deps (?) I'd like to make this module useful to others as well.
I certainly hope it gets into core... but it shouldn't have external
dependencies (like requests, pycurl, etc.). Or make those optional (i.e.
if they are available, use them, else fall back to whatever's there? A
lot of work, but maybe worthwhile?
I've a very basic prototype of the 'uri' module going but am finally
understanding what you mean by parsing issues with complex arguments.
Right now I'm accepting the following arguments to the module: url, dest,
user, password, body, method and headers.
I'm trying to figure out the best way to send headers, which the module
should see as a dict, to the module. I was hoping to be able to specify
like so:
# Example task
- name: Testing
uri: url=http://super.awesome.com user=joe password='secret' method=GET
headers='{"Content-Type": "application/atom+xml", "Second-Header-name":
"header-val"}' body='body in here'
But this doesn't work, since yaml doesn't like the ':' in the string. I've
tried a bunch of quoting variations etc... but am thinking this might not
be the best method to go at this.
Any advice on how I should approach sending this type of data to a module?
> So, I've been thinking about creating a general purpose http client
URI/URL
> module for ansible; allowing ansible to interact with web services.
Yes, please!
Cool, glad you think it'll be useful.
> While I realize this likely won't be support in core due
> to external deps (?) I'd like to make this module useful to others as
well.
I certainly hope it gets into core... but it shouldn't have external
dependencies (like requests, pycurl, etc.). Or make those optional (i.e.
if they are available, use them, else fall back to whatever's there? A
lot of work, but maybe worthwhile?
Unfortunately, At least in the short term, I'm going to use httplib2 for
this since it covers everything in httplib and urllib2 in one library.
While I'm wrapping my head around module development and programming http I
don't want to have to worry about learning and debugging two different
libraries Good news is that is works on python 2.4 and seems to be well
supported in the different distros.
* Romeo Theriault <romeo.theriault at maine.edu> [2013/02/05 14:44]:
But this doesn't work, since yaml doesn't like the ':' in the
string. I've tried a bunch of quoting variations etc... but am
thinking this might not be the best method to go at this.
Any advice on how I should approach sending this type of data to a
module?
* Romeo Theriault <romeo.theriault at maine.edu> [2013/02/05 14:44]:
> But this doesn't work, since yaml doesn't like the ':' in the
> string. I've tried a bunch of quoting variations etc... but am
> thinking this might not be the best method to go at this.
>
> Any advice on how I should approach sending this type of data to a
> module?
URI encoding?
Thanks for the response. I'm no expert on this, so by all means prove me
wrong :), but I believe URI encoding is a different thing than HTTP Header
encoding. I've been googling a bit trying to see how/if it would be
possible to encode the headers so they could be passed to the module in one
go and am coming up somewhat empty handed. The best I'm finding is this
RFC [Character Set and Language Encoding for Hypertext Transfer Protocol
(HTTP) Header Field Parameters] (https://tools.ietf.org/html/rfc5987)
which states:
By default, message header field parameters in Hypertext Transfer Protocol
(HTTP) messages cannot carry characters outside the ISO- 8859-1 character
set....
I'm thinking though, that even if this is possible, it's really not the way
for users to be interacting with the module. They'd expect to pass an
un-encoded header name and value and I believe that's what we should do.
Also, that's what the library expects
Ahh, gotcha. Neat idea. If we're going to add the ability to pass more
complex arguments to modules I think that would be clearer solution in this
case.
Ahh, gotcha. Neat idea. If we're going to add the ability to pass more
complex arguments to modules I think that would be clearer solution in this
case.
ETA is probably 1-2 weeks and then this will be possible. Have some
other features I want to finish higher up on the list (fireball
upgrades, dry run completion).
Imo - requests not working on 2.4 is a benefit as it makes 2.4 hosts go away
However, I understand the limits people are under.
Of all the http-interfacing modules i've used in python - I like python-requests the most.
urllib* make simple things difficult and difficult things only possible.
pycurl is fine - right up to the point where you need ssl support and then it is a bag-of-cats based on which ssl stack you're built against (nss, openssl or gnutls)
if 2.4 is not really an issue - I'd say use python-requests.
