Different output from URI module on different ansible versions.

Archives Ansible Project
1

Hi all.

I'm having a problem with the URI module: in two different servers, with different ansible versions, I get different output from the same uri call. I'm taking some code from the uri module docs themselves, like this:

- name: create authentication cookie
   uri:
     url: "{{ non_public_host }}/stuff"
     validate_certs: no
     user: "{{ user }}"
     password: "{{ pass }}"
     method: GET
     headers:
       Accept: "application/stuff+json"
   register: login

# and then we use later the login fact, this way:

- name: get stuff
   uri:
     url: "{{ non_public_host }}/stuff/list"
     validate_certs: no
     method: GET
     return_content: yes
     headers:
       Cookie: "{{ login.set_cookie }}"
   register: stuff_list

It's working so far, for about 2y. But when we installed a new server recently, we found the output pasted below. There's a difference in the cookies handling. v2.4 gives us the "cookies" property, while v2.3 returns "set_cookie".

# This is the output with ansible 2.4, with some strings edited for privacy reasons:
TASK [roles/stuff : debugging login] ****************************************
ok: [127.0.0.1] => {
     "msg": {
        "cache_control": "max-age=0, private, must-revalidate",
        "changed": false,
        "connection": "close",
        "content_type": "application/stuff+json; charset=utf-8",
        "cookies": {
            "JSESSIONID": "ui74vbyaobr68mj22nmr8ggl"
        },
        "date": "Tue, 15 May 2018 15:01:53 GMT",
        "etag": "\"7a9c36bb30d647628294fb324a7c0e1f\"",
        "failed": false,
        "msg": "OK (unknown bytes)",
        "redirected": false,
        "status": 200,
        "url": "https://non-public-url/stuff",
        "vary": "Accept-Encoding, User-Agent",
        "x_content_type_options": "nosniff",
        "x_frame_options": "SAMEORIGIN",
        "x_request_id": "62f27c01-aa91-4bd8-8979-05430a1c38c4",
        "x_runtime": "0.020000",
        "x_ua_compatible": "chrome=1",
        "x_xss_protection": "1; mode=block"
    }
}

# This is the output with ansible 2.3, using the same role with the same url:
TASK [roles/stuff : debugging login] ****************************************
ok: [localhost] => {
    "msg": {
        "cache_control": "max-age=0, private, must-revalidate",
        "changed": false,
        "connection": "close",
        "content_type": "application/stuff+json; charset=utf-8",
        "date": "Tue, 15 May 2018 15:05:17 GMT",
        "etag": "\"5ce7ca165672c9ac77e47d0c3617abb2\"",
        "expires": "Thu, 01 Jan 1970 00:00:00 GMT",
        "msg": "OK (unknown bytes)",
        "redirected": false,
        "set_cookie": "JSESSIONID=q5z2ud6gpvkh1cw3x6g5wf3ku;Path=/go;Expires=Tue, 29-May-2018 15:05:19 GMT;Secure;HttpOnly",
        "status": 200,
        "url": "https://non-public-url/stuff",
        "vary": "Accept-Encoding, User-Agent",
        "x_content_type_options": "nosniff",
        "x_frame_options": "SAMEORIGIN",
        "x_request_id": "105e228f-b356-4bc6-87dd-37c459dc008a",
        "x_runtime": "0.025000",
        "x_ua_compatible": "chrome=1",
        "x_xss_protection": "1; mode=block"
    }
}

I don't think this is a bug, but more like some kind of config somewhere. But neither the docs or anywhere online seems to point to any such direction like "config THIS like THIS in order to handle cookies THIS way". I didn't found it at least.

I don't want to handle this with ad-hoc checks: I would like to force always the same output from URI.
Any clue on how can I handle this?

Thanks in advance.

2

Transcript from a quick chat on Freenode ansible's IRC channel:

[12:56] <Canta_> Any clue on how can i force the URI module output format?
[12:56] <agaffney> can you be more specific?
[12:57] <Canta_> Sure. I was saying a few minutes ago:
[12:57] <Canta_> I do a call to a login service, this service gives me a cookie, and then I use the cookie for the rest of the calls. How to do this is stated in the URI module's docs.
[12:57] <Canta_> But with a server with ansible 2.3, the URI output object has the "set_cookie" property, and with ansible 2.4 it returns the "cookies" property. Even when both of them do the same request to the same server.
[12:58] <Canta_> So far, couldn't find how to config that. :frowning:
[12:58] <agaffney> config what? which field is used to contain the cookies? the change between 2.3 and 2.4 was either purposeful or unintentional, but it's probably not configurable
[13:01] <Canta_> @agaffney : yep, i would like to force the property for the cookies in the URI module output.
[13:01] <Canta_> @agaffney : If it's not configurable, then i guess there's an error in the module's docs.
[13:01] <agaffney> the docs don't even show 'set_cookie' or 'cookies' as a valid return param for the 'uri' module, but I do see an example of accessing .set_cookie from a registered var
[13:02] <Canta_> @agaffney : yeah. And that code in the example worked so far, until v2.4.
[13:05] <Canta_> @agaffney : Thing is, i've poked in the module's code, and couldn't find anything related to those properties, in any version (2.3 nor 2.4).
[13:05] <Canta_> @agaffney : So is kinda a mistery to me where does it comes from.
[13:05] <agaffney> the properties probably come from fetch_url() in module_utils.basics
[13:05] <agaffney> or thereabouts
[13:06] <Canta_> Hmmm... that's useful. I'll check its code and see if there's any significant change there. Thanks agaffney! :smiley:
[13:08] <agaffney> Canta_: https://github.com/ansible/ansible/commit/df8fde4d786e32ed7e8313d63e9a559d5f6137d3#diff-3448ba8582852f79eb0d697763e046d8 looks relevant
[13:09] <agaffney> that's where 'cookies' was added, but I have no idea where 'set_cookie' came from previously or where it went to
[13:10] <agaffney> actually, that commit message says that 'set_cookie' should still be there
[13:10] <agaffney> it's possible that it was removed in a later commit

So, I'm looking now for the origin of the "set_cookie" property. If it was deleted from the code, then the docs should be changed. Otherwise, there may be some other problem here.

3

Ended up creating an issue for this: https://github.com/ansible/ansible/issues/40278

4

My take on this is that is must be a bug.

The documentation in Ansible 2.3 says this is a stable interface, but it's has clearly changed in 2.4.
No change of the module is mention in the porting guide for 2.4.