I am trying to convert a few python scripts to julia. I wrote a simple script, in Python, wherein it asks for the username and password and uses them with Basic Authentication to request data from our company website. The script looks something like
user = getpass.getuser()
passwd = getpass.getpass(prompt='Please enter your password: ')
baseurl = 'https://www.notarealwebsite.com'
endpoint = 'rest/endpoint'
params_diction = {'some_fields':'abc'}
with requests.Session() as session:
session.auth = HTTPBasicAuth(username, password)
r = session.get(baseurl + endpoint, params=params_diction)
r.json()
I tried the following in julia:
using Base, HTTP, JSON
user = "username" # Could not find anything like python's getpass.getuser. The ENV["USER"] was not set"
passwd = Base.getpass("Yo password")
baseurl = 'https://$user:$passwd@www.notarealwebsite.com'
endpoint = "rest/endpoint"
params_diction = {'some_fields' => 'abc'}
r = HTTP.request("GET", baseurl * endpoint, ["Content-Type" => "application/json"], JSON.json(params_diction))
I get the following error:
ERROR: HTTP.ExceptionRequest.StatusError(403, "GET", "/rest/endpoitn", HTTP.Messages.Response:
"""
HTTP/1.1 403 Forbidden
X-AREQUESTID: 1317x6490724x1
X-ANODEID: node2
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
Content-Security-Policy: frame-ancestors 'self'
X-ASEN: SEN-10264611
Set-Cookie: JSESSIONID=9A21CA721A0D249015E4B843DBDF6D6C; Path=/; Secure; HttpOnly
X-Seraph-LoginReason: AUTHENTICATION_DENIED
WWW-Authenticate: OAuth realm="*************"
X-ASESSIONID: db0yvg
X-Authentication-Denied-Reason: CAPTCHA_CHALLENGE; login-url=*********
Content-Type: text/html;charset=UTF-8
Transfer-Encoding: chunked
Date: Tue, 19 Jan 2021 02:57:27 GMT
However, when I use the plaintext password in the url directly, I get the following error:
ERROR: DNSError: username, unknown node or service (EAI_NONAME)
Questions:
-
How do I pass a user’s password as an input to the HTTP.request method for basic authentication without revealing the actual password?
-
Why am I getting a DNSError if I pass the password as plaintext?
UPDATE:
From here, I found that I can use the:
x = read(secretbuffer_from_getpass, string)
to get the password string.
So this solves the 1st question.
However, now I am getting the following error:
ERROR: LoadError: HTTP.ExceptionRequest.StatusError(401, "GET", "/rest/api/2/search", HTTP.Messages.Response:
"""
HTTP/1.1 401 Unauthorized
X-AREQUESTID: 1390x6515484x10
X-ANODEID: node2
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
Content-Security-Policy: frame-ancestors 'self'
X-ASEN: SEN-10264611
X-Seraph-LoginReason: AUTHENTICATED_FAILED
WWW-Authenticate: OAuth realm=**************
Content-Type: text/html;charset=UTF-8
Transfer-Encoding: chunked
Date: Tue, 19 Jan 2021 04:10:18 GMT
Equivalent script in python works without any issues.
UPDATE:
From here, I found that I could do the following for the Basic Authorization.
using Base64
auth = Base64.base64encode(user * ":" * passwd)
r = HTTP.request("GET", baseurl * endpoint, ["Authorization" => "Basic $(auth)"], params_diction)
I get a 200 response!
So the https://username:password@www.someurl.com schema is not working.
But the response is wrong. I am supposed to get a single answer to my query. In the python requests module documentation we have:
" You often want to send some sort of data in the URL’s query string. If you were constructing the URL by hand, this data would be given as key/value pairs in the URL after a question mark, e.g. httpbin.org/get?key=val . Requests allows you to provide these arguments as a dictionary of strings, using the params keyword argument. As an example, if you wanted to pass key1=value1 and key2=value2 to httpbin.org/get , you would use the following code:"
>>> payload = {'key1': 'value1', 'key2': 'value2'}
>>> r = requests.get('https://httpbin.org/get', params=payload)
What is the equivalent of params=payload in HTTP.jl?
Per the (document)[https://juliaweb.github.io/HTTP.jl/stable/public_interface/#URIs]
" Basic Authentication options
- Basic authentication is detected automatically from the provided url’s
userinfo (in the form scheme://user:password@host ) and adds the Authorization: Basic header"
Would help if there was an example with this option.
Would like to know what I was doing wrong when I tried the https://username:password@www.someurl.com schema.
Seems to work just fine:
julia> import HTTP, Base64
julia> @async HTTP.listen("0.0.0.0", 8123) do http
auth_header = HTTP.header(http, "Authorization")
user_pass = match(r"^Basic\s+(.*)$", auth_header)
if user_pass === nothing
HTTP.setstatus(http, 401)
HTTP.startwrite(http)
else
user_pass = Base64.base64decode(user_pass[1])
HTTP.setstatus(http, 200)
HTTP.startwrite(http)
HTTP.write(http, "Basic auth: ", user_pass)
end
end
Task (runnable) @0x00007f03b2bd79d0
julia> HTTP.get("http://user:password@localhost:8123")
HTTP.Messages.Response:
"""
HTTP/1.1 200 OK
Transfer-Encoding: chunked
Basic auth: user:password"""
julia> HTTP.get("http://localhost:8123")
ERROR: HTTP.ExceptionRequest.StatusError(401, "GET", "/", HTTP.Messages.Response:
"""
HTTP/1.1 401 Unauthorized
Transfer-Encoding: chunked
"""
1 Like
@fredrikekre Thank you for the reply!
I expected for the header for basic auth to be set automatically, but according to documentation for the website whose REST APIs I am trying to use (here), I will have to set the header myself.
FINAL UPDATE:
I believe, I have this figured out. The HTTP.jl documentation had the final answer for me. Here is what the final code looks like:
user = "username"
x = Base.getpass("Enter the password")
passwd = read(x, String)
auth = Base64.base64encode(user * ":" * passwd)
baseurl = "https://user:passwd@www.notawebsite.com/"
endpoint = "rest/endpoint"
params_diction = Dict("fields"=>fields)
r = HTTP.request("GET", baseurl * endpoint, ["Authorization" => "Basic $(auth)"], query = params_diction)
json_resp = JSON.parse(String(r.body))
println(json_resp)
2 Likes