get_url always return 401 Unauthorized

HI,
im running on ansible 2.4.2 and trying to get a file from sharepoint, but always gotten 401 Unauthorized error (even i confirmed that the password is correct)

fatal: [hostname → localhost]: FAILED! => {
“changed”: false,
“dest”: “filename”,
“invocation”: {
“module_args”: {
“attributes”: null,
“backup”: null,
“checksum”: “”,
“client_cert”: null,
“client_key”: null,
“content”: null,
“delimiter”: null,
“dest”: “filename”,
“directory_mode”: null,
“follow”: false,
“force”: true,
“force_basic_auth”: true,
“group”: null,
“headers”: null,
“http_agent”: “ansible-httpget”,
“mode”: null,
“owner”: null,
“regexp”: null,
“remote_src”: null,
“selevel”: null,
“serole”: null,
“setype”: null,
“seuser”: null,
“sha256sum”: “”,
“src”: null,
“timeout”: 10,
“tmp_dest”: null,
“unsafe_writes”: null,
“url”: “http://someurlhere/filename.txt”,
“url_password”: “VALUE_SPECIFIED_IN_NO_LOG_PARAMETER”,
“url_username”: “benny”,
“use_proxy”: true,
“validate_certs”: false
}
},
“msg”: “Request failed”,
“response”: “HTTP Error 401: Unauthorized”,
“state”: “absent”,
“status_code”: 401,
“url”: “http://someurlhere/filename.txt”,
}

My playbook is rather simple:

  • name: get the lockdown script
    get_url:
    url: “http://someurlhere/filename.txt”,
    url_username: “{{ url_username }}”
    url_password: “{{ url_password }}”
    dest: filename
    force_basic_auth: yes
    force: yes
    validate_certs: no
    delegate_to: localhost

i can open the URL link from browser, and also tried using curl and it works (curl -u benny:mypassword -o /tmp/a.sh http://someurlhere/filename.txt)

Anyone has any idea what is the cause ?

beside, my password contains special character “$”, i have tried to put them under double-quote in the url_password field as well, but didnt work.

Thank you

I've seen instances where Sharepoint treated different clients
slightly different based on the user-agent string.
If you use "curl -A ansible-httpget .." to quickly rule that out.

Dick

Hi Dick,

thanks for the response.
adding curl -A ansible-httpget will result in below:

STATE: DO_DONE => WAITPERFORM handle 0x600057980; line 1795 (connection #0)

  • STATE: WAITPERFORM => PERFORM handle 0x600057980; line 1811 (connection #0)
  • HTTP 1.1 or later with persistent connection, pipelining supported
    < HTTP/1.1 401 Unauthorized
  • Server Microsoft-IIS/7.5 is not blacklisted
    < Server: Microsoft-IIS/7.5
    < SPRequestGuid: 024a709e-b9df-b0eb-e894-fb84e122f805
    < request-id: 024a709e-b9df-b0eb-e894-fb84e122f805
    < X-FRAME-OPTIONS: SAMEORIGIN
    < SPRequestDuration: 10
    < SPIisLatency: 0
    < WWW-Authenticate: Negotiate
  • NTLM handshake rejected
  • Authentication problem. Ignoring this.
    < WWW-Authenticate: NTLM
    < X-Powered-By: ASP.NET
    < MicrosoftSharePointTeamServices: 15.0.0.4569
    < X-Content-Type-Options: nosniff
    < X-MS-InvokeApp: 1; RequireReadOnly
    < Date: Tue, 12 Jun 2018 05:23:54 GMT
    < Content-Length: 0

Any way to bypass this in ansible get_url ?

Thank you

Have you tried adding
“force_basic_auth : yes” to your get_url task?

Yes, i tried to force:yes and force_basic_auth:yes
i guess the header agent was the issue, but not sure on how to overwrite it

You have two options.

User-Agent is request header information so could use
   headers: "User-Agent:<a User-Agent string>"

According to the source code you can also use
   http_agent: <a User-Agent string>

Hi Kai,
yes i have tried both that but no success

get_url:

http_agent: “Mozilla/5.0”

headers: “User-Agent:Mozilla/5.0”

on the debug mode, i can see that http_agent has changed as per what i have key in, but still gotten 401 Unauthorized.

i tried to use both parameter OR either one, but none succeed.

i guess for now, curl would be the straight forward

I can't help with Sharepoint since that is a product I don't use.

But if it's anal about the user-agent string you probably should use a valid one(one that exist in a agent).
To my knowledge no agent is using just "Mozilla/5.0".

Why don't you try with the user-agent string that curl is using.

Hi Kai,

Thanks for the quick response.
i tried use the curl user-agent, which is curl/7.59.0
both also return 401 unauthorized… … flusterring…