# How to discriminate between chunked and not chunked answers from uri **URL:** \ **Category:** Ansible Project\ **Created:** [March 11, 2022, 1:33pm UTC](https://forum.ansible.com/t/how-to-discriminate-between-chunked-and-not-chunked-answers-from-uri/36425 "2022-03-11T13:33:01Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![actionmystique](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/actionmystique/32/1678_2.png) [@actionmystique](https://forum.ansible.com/u/actionmystique)\ **Post date:** [March 11, 2022, 1:33pm UTC](https://forum.ansible.com/t/how-to-discriminate-between-chunked-and-not-chunked-answers-from-uri/36425/1 "2022-03-11T13:33:01Z")
ansible core 2.12.3 I am unable to discriminate with uri between responses which have been chunked and those which haven’t.In both cases, I get the same: - msg: OK (unknown bytes) - transfer\_encoding: chunked This happens for instance when using uri over CSR 1kv 17.3.1a with: - name: Sending GET datastores with uri ansible.builtin.uri: body: ‘’ body\_format: ‘json’ force\_basic\_auth: yes headers: Accept: application/yang-data+json Content-Type: application/yang-data+json method: GET return\_content: yes status\_code: “100,101,102,200,201,202,203,204,205,206,207,208,226,300,301,302,303,304,305,307,308” timeout: 120 url: “[https://172.21.126.182/restconf/data/ietf-netconf-monitoring:netconf-state/datastores?content=nonconfig](https://172.21.126.182/restconf/data/ietf-netconf-monitoring:netconf-state/datastores?content=nonconfig)” url\_password: password url\_username: admin validate\_certs: false register: return\_uri\_restconf\_json leads to: task path: main.yml:219 ESTABLISH LOCAL CONNECTION FOR USER: root EXEC /bin/bash -c ‘echo ~root && sleep 0’ EXEC /bin/bash -c ‘( umask 77 && mkdir -p “`echo /root/.ansible/tmp`”&& mkdir “`echo /root/.ansible/tmp/ansible-tmp-1646997734.4119265-2691638-25534722658108`” && echo ansible-tmp-1646997734.4119265-2691638-25534722658108=“`echo /root/.ansible/tmp/ansible-tmp-1646997734.4119265-2691638-25534722658108`” ) && sleep 0’ Using module file /usr/local/lib/python3.9/dist-packages/ansible/modules/uri.py PUT /root/.ansible/tmp/ansible-local-269044374\_5i0rc/tmpyvffo2y1 TO /root/.ansible/tmp/ansible-tmp-1646997734.4119265-2691638-25534722658108/AnsiballZ\_uri.py EXEC /bin/bash -c ‘chmod u+x /root/.ansible/tmp/ansible-tmp-1646997734.4119265-2691638-25534722658108/ /root/.ansible/tmp/ansible-tmp-1646997734.4119265-2691638-25534722658108/AnsiballZ\_uri.py && sleep 0’ EXEC /bin/bash -c ‘/usr/bin/python3 /root/.ansible/tmp/ansible-tmp-1646997734.4119265-2691638-25534722658108/AnsiballZ\_uri.py && sleep 0’ EXEC /bin/bash -c ‘rm -f -r /root/.ansible/tmp/ansible-tmp-1646997734.4119265-2691638-25534722658108/ \> /dev/null 2\>&1 && sleep 0’ ok: [CSR1000v-17.3.1a → localhost] =\> changed=false cache\_control: private, no-cache, must-revalidate, proxy-revalidate connection: close content: |- { “ietf-netconf-monitoring:datastores”: { “datastore”: [ { “name”: “running”, “tailf-netconf-monitoring:transaction-id”: “1646-983921-976163” }, { “name”: “candidate” } ] } } content\_type: application/yang-data+json cookies: {} cookies\_string: ‘’ date: Fri, 11 Mar 2022 11:22:14 GMT elapsed: 0 invocation: module\_args: attributes: null body: ‘’ body\_format: json ca\_path: null client\_cert: null client\_key: null creates: null dest: null follow\_redirects: safe force: false force\_basic\_auth: true group: null headers: Accept: application/yang-data+json Content-Type: application/yang-data+json http\_agent: ansible-httpget method: GET mode: null owner: null remote\_src: false removes: null return\_content: true selevel: null serole: null setype: null seuser: null src: null status\_code: - 100 - 101 - 102 - 200 - 201 - 202 - 203 - 204 - 205 - 206 - 207 - 208 - 226 - 300 - 301 - 302 - 303 - 304 - 305 - 307 - 308 timeout: 120 unix\_socket: null unredirected\_headers: unsafe\_writes: false url: [https://172.21.126.182/restconf/data/ietf-netconf-monitoring:netconf-state/datastores?content=nonconfig](https://172.21.126.182/restconf/data/ietf-netconf-monitoring:netconf-state/datastores?content=nonconfig) url\_password: VALUE\_SPECIFIED\_IN\_NO\_LOG\_PARAMETER url\_username: admin use\_gssapi: false use\_proxy: true validate\_certs: false json: ietf-netconf-monitoring:datastores: datastore: - name: running tailf-netconf-monitoring:transaction-id: 1646-983921-976163 - name: candidate msg: OK (unknown bytes) pragma: no-cache redirected: false server: openresty status: 200 transfer\_encoding: chunked url: [https://172.21.126.182/restconf/data/ietf-netconf-monitoring:netconf-state/datastores?content=nonconfig](https://172.21.126.182/restconf/data/ietf-netconf-monitoring:netconf-state/datastores?content=nonconfig) We can see that the response has NOT been chunked. In another example, the response has been chunked with: - name: Sending GET netconf-state with uri ansible.builtin.uri: body: ‘’ body\_format: ‘json’ force\_basic\_auth: yes headers: Accept: application/yang-data+json Content-Type: application/yang-data+json method: GET return\_content: yes status\_code: “100,101,102,200,201,202,203,204,205,206,207,208,226,300,301,302,303,304,305,307,308” timeout: 120 url: “[https://172.21.126.182/restconf/data/ietf-netconf-monitoring:netconf-state?content=nonconfig](https://172.21.126.182/restconf/data/ietf-netconf-monitoring:netconf-state?content=nonconfig)” url\_password: password url\_username: admin validate\_certs: false register: return\_uri\_restconf\_json which leads to a really chunked answer: task path: main.yml:219 ESTABLISH LOCAL CONNECTION FOR USER: root EXEC /bin/bash -c ‘echo ~root && sleep 0’ EXEC /bin/bash -c ‘( umask 77 && mkdir -p “`echo /root/.ansible/tmp`”&& mkdir “`echo /root/.ansible/tmp/ansible-tmp-1646997777.3538132-2700276-93994275105673`” && echo ansible-tmp-1646997777.3538132-2700276-93994275105673=“`echo /root/.ansible/tmp/ansible-tmp-1646997777.3538132-2700276-93994275105673`” ) && sleep 0’ Using module file /usr/local/lib/python3.9/dist-packages/ansible/modules/uri.py PUT /root/.ansible/tmp/ansible-local-269044374\_5i0rc/tmps9ux33ea TO /root/.ansible/tmp/ansible-tmp-1646997777.3538132-2700276-93994275105673/AnsiballZ\_uri.py EXEC /bin/bash -c ‘chmod u+x /root/.ansible/tmp/ansible-tmp-1646997777.3538132-2700276-93994275105673/ /root/.ansible/tmp/ansible-tmp-1646997777.3538132-2700276-93994275105673/AnsiballZ\_uri.py && sleep 0’ EXEC /bin/bash -c ‘/usr/bin/python3 /root/.ansible/tmp/ansible-tmp-1646997777.3538132-2700276-93994275105673/AnsiballZ\_uri.py && sleep 0’ EXEC /bin/bash -c ‘rm -f -r /root/.ansible/tmp/ansible-tmp-1646997777.3538132-2700276-93994275105673/ \> /dev/null 2\>&1 && sleep 0’ ok: [CSR1000v-17.3.1a → localhost] =\> changed=false cache\_control: private, no-cache, must-revalidate, proxy-revalidate connection: close content: |- { “ietf-netconf-monitoring:netconf-state”: { “capabilities”: { … }, “datastores”: { “datastore”: [ { “name”: “running”, “tailf-netconf-monitoring:transaction-id”: “1646-983921-976163” }, { “name”: “candidate” } ] }, “schemas”: { … }, “sessions”: { “session”: [ { “session-id”: 26, “transport”: “tailf-netconf-monitoring:rest-http”, “username”: “admin”, “source-host”: “172.21.0.1”, “login-time”: “2022-03-11T11:22:57+00:00”, “tailf-netconf-monitoring:transaction”: [ { content\_type: application/yang-data+json cookies: {} cookies\_string: ‘’ date: Fri, 11 Mar 2022 11:22:57 GMT elapsed: 0 invocation: module\_args: attributes: null body: ‘’ body\_format: json ca\_path: null client\_cert: null client\_key: null creates: null dest: null follow\_redirects: safe force: false force\_basic\_auth: true group: null headers: Accept: application/yang-data+json Content-Type: application/yang-data+json http\_agent: ansible-httpget method: GET mode: null owner: null remote\_src: false removes: null return\_content: true selevel: null serole: null setype: null seuser: null src: null status\_code: - 100 - 101 - 102 - 200 - 201 - 202 - 203 - 204 - 205 - 206 - 207 - 208 - 226 - 300 - 301 - 302 - 303 - 304 - 305 - 307 - 308 timeout: 120 unix\_socket: null unredirected\_headers: unsafe\_writes: false url: [https://172.21.126.182/restconf/data/ietf-netconf-monitoring:netconf-state?content=nonconfig](https://172.21.126.182/restconf/data/ietf-netconf-monitoring:netconf-state?content=nonconfig) url\_password: VALUE\_SPECIFIED\_IN\_NO\_LOG\_PARAMETER url\_username: admin use\_gssapi: false use\_proxy: true validate\_certs: false msg: OK (unknown bytes) pragma: no-cache redirected: false server: openresty status: 200 transfer\_encoding: chunked url: [https://172.21.126.182/restconf/data/ietf-netconf-monitoring:netconf-state?content=nonconfig](https://172.21.126.182/restconf/data/ietf-netconf-monitoring:netconf-state?content=nonconfig) Any suggestion about how to differentiate between both types of responses? ---
**Author:** ![sivel](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/sivel/32/1198_2.png) [@sivel](https://forum.ansible.com/u/sivel)\ **Post date:** [March 11, 2022, 3:16pm UTC](https://forum.ansible.com/t/how-to-discriminate-between-chunked-and-not-chunked-answers-from-uri/36425/2 "2022-03-11T15:16:30Z")
Python 3 should be attempting to read all chunks if the transfer-encoding is chunked when a single `.read()` is called. I would not expect it to fail to fetch all chunks. As such, even if `transfer_encoding: chunked` is displayed, python should have still fetched the full content. So, to more specifically answer your question, if you see `transfer_encoding: chunked`, then the response was chunked, but python has attempted to fetch all chunks. I suppose to determine if you have a partially read chunk, and not the full response, you would have to look at whether the `.json` key exists on the module response. ---
**Author:** ![actionmystique](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/actionmystique/32/1678_2.png) [@actionmystique](https://forum.ansible.com/u/actionmystique)\ **Post date:** [March 11, 2022, 4:52pm UTC](https://forum.ansible.com/t/how-to-discriminate-between-chunked-and-not-chunked-answers-from-uri/36425/3 "2022-03-11T16:52:44Z")
Thanks. But I understand from your answer that really chunked answers should not happen under normal circumstances. Since “Python 3 should be attempting to read all chunks”, I tried to set the timeout to a high value (3600) to eliminate that potential explanation for the chunked answer. However, it is still happening. Should I report this a bug or can you suggest anything else? Also, by default (if no other headers are set), is it possible to receive something else than ‘transfer\_encoding: chunked’ from the server for body\_format: ‘json’ or body\_format: ‘raw’? ---
**Author:** ![sivel](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/sivel/32/1198_2.png) [@sivel](https://forum.ansible.com/u/sivel)\ **Post date:** [March 11, 2022, 5:13pm UTC](https://forum.ansible.com/t/how-to-discriminate-between-chunked-and-not-chunked-answers-from-uri/36425/4 "2022-03-11T17:13:27Z")
I wouldn’t know where an appropriate place to file a bug would be. Looking at the CPython code for http.client, `.read()` should be fully consuming all of the chunks. [https://github.com/python/cpython/blob/3.10/Lib/http/client.py#L577](https://github.com/python/cpython/blob/3.10/Lib/http/client.py#L577) As such, I doubt we would do anything in ansible-core to deal with this. There is no way for the client to instruct the server to not send a chunked response. A chunked response is used when the Content-Length isn’t known at the time the response is sent back to the client. ---
**Author:** ![actionmystique](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/actionmystique/32/1678_2.png) [@actionmystique](https://forum.ansible.com/u/actionmystique)\ **Post date:** [March 11, 2022, 5:29pm UTC](https://forum.ansible.com/t/how-to-discriminate-between-chunked-and-not-chunked-answers-from-uri/36425/5 "2022-03-11T17:29:54Z")
But how can the Content-Length be unknown at the time the response is sent back to the client for small unchunked answers (case n°1 of my first post)? ---
**Author:** ![actionmystique](https://sea1.discourse-cdn.com/ansible/user_avatar/forum.ansible.com/actionmystique/32/1678_2.png) [@actionmystique](https://forum.ansible.com/u/actionmystique)\ **Post date:** [March 11, 2022, 5:53pm UTC](https://forum.ansible.com/t/how-to-discriminate-between-chunked-and-not-chunked-answers-from-uri/36425/6 "2022-03-11T17:53:47Z")
I have filed the issue [here](https://github.com/ansible/ansible/issues/77262)