# Contemplating a URI Module **URL:** \ **Category:** Ansible Project\ **Created:** [February 4, 2013, 8:48pm UTC](https://forum.ansible.com/t/contemplating-a-uri-module/12292 "2013-02-04T20:48:32Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![Romeo\_Theriault](https://avatars.discourse-cdn.com/v4/letter/r/839c29/32.png) [@Romeo\_Theriault](https://forum.ansible.com/u/Romeo_Theriault)\ **Post date:** [February 4, 2013, 8:48pm UTC](https://forum.ansible.com/t/contemplating-a-uri-module/12292/1 "2013-02-04T20:48:32Z")
So, I’ve been thinking about creating a general purpose http client URI/URL module for ansible; allowing ansible to interact with web services. Some example use cases I’ve thought of: - Create a JIRA ticket on some condition using JIRA’s RESTful API - Delete/Update said JIRA ticket - Add a host to ‘your-favorite’ monitoring/CMDB system via it’s web API. (I use zabbix which has such an API.) - Delete said host from the monitoring/CMDB system. - Sending ansible ‘setup’ module json output to some yet created webservice for host-inventory cataloging. - Generally, just allowing ansible to interact with web-services. Still in the planning stages but the biggest problem I see right now is that the standard & included HTTP client libraries in python have some rather big holes. urllib2 only supports GET and POST HTTP methods. httplib doesn’t appear to support authentication, proxies, compression, cookies or redirects. That leaves either non-standard python http client libraries or curl. Sticking with python will certainly make the job easier. The two popular python http client libs I see are cover all/most features are: - httplib2 (big bonus in that it runs on python 2.4) - requests (doesn’t support python 2.4) I’m tempted to use httplib2 for this since it’ll be sticking with python but also support python 2.4. But before I start working on this though I’d like to get feedback from others to see if I might be missing something glaringly obvious. While I realize this likely won’t be support in core due to external deps (?) I’d like to make this module useful to others as well. Thanks! ---
**Author:** ![Michael\_DeHaan](https://avatars.discourse-cdn.com/v4/letter/m/c4cdca/32.png) [@Michael\_DeHaan](https://forum.ansible.com/u/Michael_DeHaan)\ **Post date:** [February 4, 2013, 8:57pm UTC](https://forum.ansible.com/t/contemplating-a-uri-module/12292/2 "2013-02-04T20:57:07Z")
Sounds fantastic for moving us closer to general purpose orchestration. Anything as long as it is not requests -- \*it\* has too many holes in various cases. urllib2 with httplib DELETE for odd methods was, I think, how I did it in the past ... can't remember? (I know Seth has some opinions.) ---
**Author:** ![Brian\_Coca1](https://avatars.discourse-cdn.com/v4/letter/b/278dde/32.png) [@Brian\_Coca1](https://forum.ansible.com/u/Brian_Coca1)\ **Post date:** [February 4, 2013, 10:17pm UTC](https://forum.ansible.com/t/contemplating-a-uri-module/12292/3 "2013-02-04T22:17:22Z")
I'm using pycurl for something similar but this requires itself and curl as dependencies ---
**Author:** ![Michael\_DeHaan](https://avatars.discourse-cdn.com/v4/letter/m/c4cdca/32.png) [@Michael\_DeHaan](https://forum.ansible.com/u/Michael_DeHaan)\ **Post date:** [February 4, 2013, 10:32pm UTC](https://forum.ansible.com/t/contemplating-a-uri-module/12292/4 "2013-02-04T22:32:59Z")
BTW, for POSTS and PUTS, do you see that going from a template in many cases? with\_template could be very good for that. XML should not /usually/ care about whitespace. I also want to make AnsibleModuleCommon transparently start using JSON which will cleanup some parsing dependencies anyway... and enable what some people have been wanting about sending complex arguments to modules. ---
**Author:** ![Romeo\_Theriault](https://avatars.discourse-cdn.com/v4/letter/r/839c29/32.png) [@Romeo\_Theriault](https://forum.ansible.com/u/Romeo_Theriault)\ **Post date:** [February 4, 2013, 11:26pm UTC](https://forum.ansible.com/t/contemplating-a-uri-module/12292/5 "2013-02-04T23:26:14Z")
> BTW, for POSTS and PUTS, do you see that going from a template in many > cases? with\_template could be very good for that. Yes, I imagined templates would be quite useful in generating the appropriate payload to send. Wasn't aware of 'with\_template', thanks for pointing that out. > XML should not /usually/ care about whitespace. > > I also want to make AnsibleModuleCommon transparently start using JSON > which will cleanup some parsing dependencies anyway... and enable what > some people have been wanting about sending complex arguments to > modules. Ok, I'll keep this in mind as I proceed. ---
**Author:** ![jpmens1](https://avatars.discourse-cdn.com/v4/letter/j/e9c0ed/32.png) [@jpmens1](https://forum.ansible.com/u/jpmens1)\ **Post date:** [February 5, 2013, 6:35am UTC](https://forum.ansible.com/t/contemplating-a-uri-module/12292/6 "2013-02-05T06:35:17Z")
> So, I've been thinking about creating a general purpose http client URI/URL > module for ansible; allowing ansible to interact with web services. Yes, please! > While I realize this likely won't be support in core due > to external deps (?) I'd like to make this module useful to others as well. I certainly hope it gets into core... but it shouldn't have external dependencies (like requests, pycurl, etc.). Or make those optional (i.e. if they are available, use them, else fall back to whatever's there? A lot of work, but maybe worthwhile? 🙂         -JP ---
**Author:** ![Romeo\_Theriault](https://avatars.discourse-cdn.com/v4/letter/r/839c29/32.png) [@Romeo\_Theriault](https://forum.ansible.com/u/Romeo_Theriault)\ **Post date:** [February 6, 2013, 12:44am UTC](https://forum.ansible.com/t/contemplating-a-uri-module/12292/7 "2013-02-06T00:44:34Z")
I've a very basic prototype of the 'uri' module going but am finally understanding what you mean by parsing issues with complex arguments. Right now I'm accepting the following arguments to the module: url, dest, user, password, body, method and headers. I'm trying to figure out the best way to send headers, which the module should see as a dict, to the module. I was hoping to be able to specify like so: # Example task - name: Testing    uri: url=[http://super.awesome.com](http://super.awesome.com) user=joe password='secret' method=GET headers='{"Content-Type": "application/atom+xml", "Second-Header-name": "header-val"}' body='body in here' But this doesn't work, since yaml doesn't like the ':' in the string. I've tried a bunch of quoting variations etc... but am thinking this might not be the best method to go at this. Any advice on how I should approach sending this type of data to a module? Thanks, Romeo ---
**Author:** ![Romeo\_Theriault](https://avatars.discourse-cdn.com/v4/letter/r/839c29/32.png) [@Romeo\_Theriault](https://forum.ansible.com/u/Romeo_Theriault)\ **Post date:** [February 6, 2013, 12:50am UTC](https://forum.ansible.com/t/contemplating-a-uri-module/12292/8 "2013-02-06T00:50:16Z")
> \> So, I've been thinking about creating a general purpose http client > URI/URL > \> module for ansible; allowing ansible to interact with web services. > > Yes, please! Cool, glad you think it'll be useful. > \> While I realize this likely won't be support in core due > \> to external deps (?) I'd like to make this module useful to others as > well. > > I certainly hope it gets into core... but it shouldn't have external > dependencies (like requests, pycurl, etc.). Or make those optional (i.e. > if they are available, use them, else fall back to whatever's there? A > lot of work, but maybe worthwhile? 🙂 Unfortunately, At least in the short term, I'm going to use httplib2 for this since it covers everything in httplib and urllib2 in one library. While I'm wrapping my head around module development and programming http I don't want to have to worry about learning and debugging two different libraries 🙂 Good news is that is works on python 2.4 and seems to be well supported in the different distros. ---
**Author:** ![Michael\_DeHaan](https://avatars.discourse-cdn.com/v4/letter/m/c4cdca/32.png) [@Michael\_DeHaan](https://forum.ansible.com/u/Michael_DeHaan)\ **Post date:** [February 6, 2013, 3:00am UTC](https://forum.ansible.com/t/contemplating-a-uri-module/12292/9 "2013-02-06T03:00:11Z")
We really need to implement complex args to modules to make this tenable. That being said, we are doing that, but headers is a hash, it should not be a eviley quoted string to make it accessible. All of this will be in work, as I've discussed a few times in the past, so stay tuned 🙂 --Michael ---
**Author:** ![Romeo\_Theriault](https://avatars.discourse-cdn.com/v4/letter/r/839c29/32.png) [@Romeo\_Theriault](https://forum.ansible.com/u/Romeo_Theriault)\ **Post date:** [February 6, 2013, 4:59am UTC](https://forum.ansible.com/t/contemplating-a-uri-module/12292/10 "2013-02-06T04:59:48Z")
Alright, thanks. I'll stay tuned. In the mean time I'll keep working on other aspects of the module. ---
**Author:** ![jpmens1](https://avatars.discourse-cdn.com/v4/letter/j/e9c0ed/32.png) [@jpmens1](https://forum.ansible.com/u/jpmens1)\ **Post date:** [February 6, 2013, 8:33am UTC](https://forum.ansible.com/t/contemplating-a-uri-module/12292/11 "2013-02-06T08:33:46Z")
> Good news is that is works on python 2.4 and seems to be well > supported in the different distros. That's all that matters, at least to me. Good stuff.         -JP ---
**Author:** ![Chamberlain\_Darren](https://avatars.discourse-cdn.com/v4/letter/c/ee7513/32.png) [@Chamberlain\_Darren](https://forum.ansible.com/u/Chamberlain_Darren)\ **Post date:** [February 6, 2013, 4:35pm UTC](https://forum.ansible.com/t/contemplating-a-uri-module/12292/12 "2013-02-06T16:35:52Z")
\* Romeo Theriault \ [2013/02/05 14:44]: > But this doesn't work, since yaml doesn't like the ':' in the > string. I've tried a bunch of quoting variations etc... but am > thinking this might not be the best method to go at this. > > Any advice on how I should approach sending this type of data to a > module? URI encoding? ---
**Author:** ![Romeo\_Theriault](https://avatars.discourse-cdn.com/v4/letter/r/839c29/32.png) [@Romeo\_Theriault](https://forum.ansible.com/u/Romeo_Theriault)\ **Post date:** [February 6, 2013, 11:47pm UTC](https://forum.ansible.com/t/contemplating-a-uri-module/12292/13 "2013-02-06T23:47:54Z")
> \* Romeo Theriault \ [2013/02/05 14:44]: > \> But this doesn't work, since yaml doesn't like the ':' in the > \> string. I've tried a bunch of quoting variations etc... but am > \> thinking this might not be the best method to go at this. > \> > \> Any advice on how I should approach sending this type of data to a > \> module? > > URI encoding? Thanks for the response. I'm no expert on this, so by all means prove me wrong :), but I believe URI encoding is a different thing than HTTP Header encoding. I've been googling a bit trying to see how/if it would be possible to encode the headers so they could be passed to the module in one go and am coming up somewhat empty handed. The best I'm finding is this RFC [Character Set and Language Encoding for Hypertext Transfer Protocol (HTTP) Header Field Parameters] ([https://tools.ietf.org/html/rfc5987](https://tools.ietf.org/html/rfc5987)) which states: By default, message header field parameters in Hypertext Transfer Protocol > (HTTP) messages cannot carry characters outside the ISO- 8859-1 character > set.... I'm thinking though, that even if this is possible, it's really not the way for users to be interacting with the module. They'd expect to pass an un-encoded header name and value and I believe that's what we should do. Also, that's what the library expects 🙂 ---
**Author:** ![Chamberlain\_Darren](https://avatars.discourse-cdn.com/v4/letter/c/ee7513/32.png) [@Chamberlain\_Darren](https://forum.ansible.com/u/Chamberlain_Darren)\ **Post date:** [February 7, 2013, 2:17am UTC](https://forum.ansible.com/t/contemplating-a-uri-module/12292/14 "2013-02-07T02:17:35Z")
\* Romeo Theriault \ [2013/02/06 13:47]: ---
**Author:** ![Romeo\_Theriault](https://avatars.discourse-cdn.com/v4/letter/r/839c29/32.png) [@Romeo\_Theriault](https://forum.ansible.com/u/Romeo_Theriault)\ **Post date:** [February 7, 2013, 3:36am UTC](https://forum.ansible.com/t/contemplating-a-uri-module/12292/15 "2013-02-07T03:36:47Z")
Ahh, gotcha. Neat idea. If we're going to add the ability to pass more complex arguments to modules I think that would be clearer solution in this case. ---
**Author:** ![Michael\_DeHaan](https://avatars.discourse-cdn.com/v4/letter/m/c4cdca/32.png) [@Michael\_DeHaan](https://forum.ansible.com/u/Michael_DeHaan)\ **Post date:** [February 7, 2013, 12:37pm UTC](https://forum.ansible.com/t/contemplating-a-uri-module/12292/16 "2013-02-07T12:37:51Z")
> Ahh, gotcha. Neat idea. If we're going to add the ability to pass more > complex arguments to modules I think that would be clearer solution in this > case. ETA is probably 1-2 weeks and then this will be possible. Have some other features I want to finish higher up on the list (fireball upgrades, dry run completion). ---
**Author:** ![Seth\_Vidal](https://avatars.discourse-cdn.com/v4/letter/s/7ba0ec/32.png) [@Seth\_Vidal](https://forum.ansible.com/u/Seth_Vidal)\ **Post date:** [February 7, 2013, 5:19pm UTC](https://forum.ansible.com/t/contemplating-a-uri-module/12292/17 "2013-02-07T17:19:25Z")
Imo - requests not working on 2.4 is a benefit as it makes 2.4 hosts go away 🙂 However, I understand the limits people are under. Of all the http-interfacing modules i've used in python - I like python-requests the most. urllib\* make simple things difficult and difficult things only possible. pycurl is fine - right up to the point where you need ssl support and then it is a bag-of-cats based on which ssl stack you're built against (nss, openssl or gnutls) if 2.4 is not really an issue - I'd say use python-requests. that's my 2c nothing more -sv ---
**Author:** ![Romeo\_Theriault](https://avatars.discourse-cdn.com/v4/letter/r/839c29/32.png) [@Romeo\_Theriault](https://forum.ansible.com/u/Romeo_Theriault)\ **Post date:** [February 7, 2013, 7:58pm UTC](https://forum.ansible.com/t/contemplating-a-uri-module/12292/18 "2013-02-07T19:58:21Z")
Sounds good. Thanks. I continue to hack away at other aspects of the module. ---
**Author:** ![Brian\_Dunbar](https://avatars.discourse-cdn.com/v4/letter/b/4491bb/32.png) [@Brian\_Dunbar](https://forum.ansible.com/u/Brian_Dunbar)\ **Post date:** [March 18, 2014, 8:39pm UTC](https://forum.ansible.com/t/contemplating-a-uri-module/12292/19 "2014-03-18T20:39:07Z")
I came here looking exactly for this. Is this uri module ready for prime time? Or even a beta? What I’ve facing is managing an elasticsearch (ES) cluster. ES likes to be managed using http like so … - Take a snapshot curl -XPUT "localhost:9200/\_snapshot/my\_backup/snapshot\_1? Which I can do with a lot of shell in playbooks but - as you know - that gets messy in a big hurry. ---
**Author:** ![sivel](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/sivel/32/1198_2.png) [@sivel](https://forum.ansible.com/u/sivel)\ **Post date:** [March 18, 2014, 8:48pm UTC](https://forum.ansible.com/t/contemplating-a-uri-module/12292/20 "2014-03-18T20:48:35Z")
Brian, This thread is over a year old. There is a URI module and you can find the docs at [http://docs.ansible.com/uri\_module.html](http://docs.ansible.com/uri_module.html) [Next page](https://forum.ansible.com/t/contemplating-a-uri-module/12292.md?page=2)