Hi! I’ve been using Ansible for about a month. I recently filed my first PR, and I’m hoping to continue the discussion now that it’s been merged: https://github.com/ansible/ansible-modules-core/pull/1634
The ‘get_url’ module (and some others) use Ansible’s ‘fetch_url’, based on urllib2. On the other hand, Ansible’s ‘uri’ module uses httplib2. I’m in an environment with a lot of networking layers (VPN, squid proxy, SOCKS proxy, and SSH tunnels), and I’ve seen their behavior diverge on a few occasions. It would be nice to trust that if ‘get_url’ works on a machine, ‘uri’ will too.
Is there anything blocking a refactor to use ‘fetch_url’ instead of httplib2? The module’s comments don’t explain why a different implementation was used. Alternately, if it’s not feasible to use ‘fetch_url’, would requests be a better dependency than httplib2? The httplib2 code in use seems to be exactly the kind of boilerplate that requests takes care of.
Possibly relevant: AFAICT this is the only module that use httplib2, which would allow removing it as an Ansible dependency.
Thanks,
-James M.
I'm not aware of a reason other than they were written at different times.
I would love a refactor to use fetch_url instead. That would make it
so that we don't have to require any additional modules. This is a
complaint that we've had to field before and it would be nice to be
able to resolve it.
requests would be a distant second choice but if we did have to do
that because fetch_url() wasn't feasible, it would be preferable to
httplib2 because requests is already installed on more machines than
httplib2. However, you have to support both the old requests API and
the more modern one because different distros have older and newer
versions. The base modules like uri have to remain python-2.4
compatible to run on RHEL/CentOS5. So we'd have to support the
version of requests that's available on those platforms (likely
through EPEL) in addition to the newer version that will be available
on newer distributions.
(So hopefully, fetch_url() will be usable here 
-Toshio
You can find my original, ‘Contemplating a URI module’ email to the list back from back in 2013, here:
https://groups.google.com/d/msg/ansible-project/A0ND645fc2Y/_BMtSy0G7iAJ
which covers the reasons I went with httplib2.
As to why I didn’t go with requests, it had some holes in it’s coverage of python 2.4 and Michael requested that I not use it.
If I was mistaken in any of my reasoning and it makes sense to port it to ‘fetch_url’ then it’s ok with me 
Romeo
It seems that the same reasoning still applies, urllib does not
provide all the methods, requests ... well ... its much better now but
considering the versions out in the wild and that they rewrote the
interface at one point its just going to be one issue after another.
It seems httplib might be the least bad option.
You can find my original, ‘Contemplating a URI module’ email to the list back from back in 2013, here:
https://groups.google.com/d/msg/ansible-project/A0ND645fc2Y/_BMtSy0G7iAJ
which covers the reasons I went with httplib2.
As to why I didn’t go with requests, it had some holes in it’s coverage of python 2.4 and Michael requested that I not use it.
If I was mistaken in any of my reasoning and it makes sense to port it to ‘fetch_url’ then it’s ok with me
Probably not mistaken per se ; jimi-c added support for additional http verbs to fetch_url at some point. There could be done missing, though. Have to let him know if that’s the case.
https://github.com/ansible/ansible/blob/devel/lib/ansible/module_utils/urls.py
(Fetch_url builds on urllib2 to provide us with a function we can call from within ansible to make http requests.)
-Toshio