Pkg: Proxy Authentication Required (Received HTTP code 407 from proxy after CONNECT)

[Keluaa]

Hello,

I am currently working with Julia 1.7.2 on a linux machine through ssh which requires a proxy to access the internet.

By configuring the environment variables “http_proxy” and “https_proxy” to http://usr:@some_address:port/ (there is no password), I was able to clone repositories with git in the command line and make requests using wget or curl.

However I was not able to install any Julia packages because of this error:

(v1.7) pkg> add JSON
    Updating registry at `~/.julia/registries/General.toml`
┌ Warning: could not download https://pkg.julialang.org/registries
│   exception = HTTP/1.1 407 Proxy Authentication Required (Received HTTP code 407 from proxy after CONNECT) while requesting https://pkg.julialang.org/registries
└ @ Pkg.Registry /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.7/Pkg/src/Registry/Registry.jl:82
   Resolving package versions...
     Cloning [682c06a0-de6a-54ab-a142-c8b1cf79cde6] JSON from https://github.com/JuliaIO/JSON.jl.git
ERROR: failed to clone from https://github.com/JuliaIO/JSON.jl.git, error: GitError(Code:ERROR, Class:HTTP, proxy authentication required but no callback set)

After some research I found this topic: Can't install packages on corporate pc · Issue #2516 · JuliaLang/Pkg.jl · GitHub
Setting JULIA_PKG_USE_CLI_GIT to true circumvents the issue, but I am still stuck with this problem: “Proxy Authentication Required (Received HTTP code 407 from proxy after CONNECT)” which occurs when trying to update the package registry.

On top of that I am quite sure that this issue is also responsible for this error :

(v1.7) pkg> add CUDA
...
  Downloaded artifact: LLVMExtra
ERROR: Unable to automatically install 'LLVMExtra' from '/home/.julia/packages/LLVMExtra_jll/y3M2E/Artifacts.toml'
Stacktrace:
  [1] error(s::String)
    @ Base ./error.jl:33
  [2] ensure_artifact_installed(name::String, meta::Dict{String, Any}, artifacts_toml::String; platform::Base.BinaryPlatforms.Platform, verbose::Bool, quiet_download::Bool, io::Base.TTY)
    @ Pkg.Artifacts /home/julia-1.7.2/share/julia/stdlib/v1.7/Pkg/src/Artifacts.jl:441
  [3] download_artifacts(env::Pkg.Types.EnvCache; platform::Base.BinaryPlatforms.Platform, julia_version::VersionNumber, verbose::Bool, io::Base.TTY)
    @ Pkg.Operations /home/julia-1.7.2/share/julia/stdlib/v1.7/Pkg/src/Operations.jl:617
  [4] add(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}, new_git::Set{Base.UUID}; preserve::Pkg.Types.PreserveLevel, platform::Base.BinaryPlatforms.Platform)
    @ Pkg.Operations /home/julia-1.7.2/share/julia/stdlib/v1.7/Pkg/src/Operations.jl:1182
...

I cannot debug the installation of the artifact with “DebugArtifacts” since it is not in the registry currently on the machine (which I copied from another machine with proper internet access):

julia> Pkg.add("DebugArtifacts")
ERROR: The following package names could not be resolved:
 * DebugArtifacts (not found in project, manifest or registry)

I am aware that this is most likely a proxy configuration issue, but I do not know which things to configure in order for Julia to work properly.

[Keluaa]

By further exploring the issue, I tracked down the problem to the fact that Julia only uses the Basic authentication type, while my proxy uses Negotiate (see the docs).

Passing verbose=true as an option to Downloads.jl, we can see that I correctly passes a valid header for Basic authentication:

julia> Downloads.download("https://pkg.julialang.org/registries"; verbose=true)
* Uses proxy env variable no_proxy == 'localhost,127.0.0.0/8,::1'
* Uses proxy env variable https_proxy == 'http://usr:@some_address:port/'
* Couldn't find host pkg.julialang.org in the .netrc file; using defaults
*   Trying ***.***.**.***:port...
* Connected to some_address (***.***.**.***) port (#0)
* allocate connect buffer!
* Establish HTTP proxy tunnel to pkg.julialang.org:443
* Proxy auth using Basic with user 'user'
> CONNECT pkg.julialang.org:443 HTTP/1.1
Host: pkg.julialang.org:443
Proxy-Authorization: Basic XXXXX
User-Agent: curl/7.73.0 julia/1.7
Proxy-Connection: Keep-Alive

< HTTP/1.1 407 Proxy Authentication Required
< Server: squid
< Mime-Version: 1.0
< Date: Wed, 27 Apr 2022 14:45:42 GMT
< Content-Type: text/html;charset=utf-8
< Content-Length: 3642
< X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0
< Vary: Accept-Language
< Content-Language: en
< Proxy-Authenticate: Negotiate
< X-Cache: MISS from some_address
< X-Cache-Lookup: NONE from some_address:port
< Via: 1.1 some_address (squid)
< Connection: keep-alive
< 
* Ignore 3642 bytes of response-body
* Received HTTP code 407 from proxy after CONNECT
* CONNECT phase completed!
* Closing connection 0
ERROR: HTTP/1.1 407 Proxy Authentication Required (Received HTTP code 407 from proxy after CONNECT) while requesting https://pkg.julialang.org/registries

But the server expects another type of authentication Proxy-Authenticate: Negotiate, so the connection is refused.

However when doing the same request with curl in the terminal, it correctly loads my ~/.curlrc config file which specifies to use Negotiate:

proxy = some_proxy:port
--proxy-negotiate
--proxy-user "user:"
-k

Therefore curl is able to perform the request:

$ curl -v https://pkg.julialang.org/registries
* About to connect() to proxy some_proxy port 'port' (#0)
*   Trying ***.***.**.***...
* Connected to some_proxy (***.***.**.***) port 'port' (#0)
* Establish HTTP proxy tunnel to pkg.julialang.org:443
> CONNECT pkg.julialang.org:443 HTTP/1.1
> Host: pkg.julialang.org:443
> User-Agent: curl/7.29.0
> Proxy-Connection: Keep-Alive
> 
< HTTP/1.1 407 Proxy Authentication Required
< Server: squid
< Mime-Version: 1.0
< Date: Thu, 28 Apr 2022 07:27:14 GMT
< Content-Type: text/html;charset=utf-8
< Content-Length: 3577
< X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0
< Vary: Accept-Language
< Content-Language: en
< Proxy-Authenticate: Negotiate
< X-Cache: MISS from some_address
< X-Cache-Lookup: NONE from some_address:port
< Via: 1.1 some_address (squid)
< Connection: keep-alive
< 
* Ignore 3577 bytes of response-body
* TUNNEL_STATE switched to: 0
* Establish HTTP proxy tunnel to pkg.julialang.org:443
* Proxy auth using GSS-Negotiate with user 'user'
> CONNECT pkg.julialang.org:443 HTTP/1.1
> Host: pkg.julialang.org:443
> Proxy-Authorization: Negotiate auth_dataXXXXXXXXXXXXX
> User-Agent: curl/7.29.0
> Proxy-Connection: Keep-Alive
> 
< HTTP/1.1 200 Connection established
...

Then, my problem becomes: how to configure Julia to use those options in the ‘~/.curlrc’ file?
I know that libcurl doesn’t read this file, so then how can I pass those parameters to Julia?
Does Julia (or Downloads.jl) even support authentication types other than Basic?

[Keluaa]

From what I could gather from reading the code of Pkg.jl and Download, it seems that there is no support for proxy authentication at all when downloading files.

Therefore I tried overriding Downloads.download function, which is used by the package manager for all (I think?) downloads.

Here is my solution, I hope that it could be useful to someone in the future:

using Downloads
import Downloads: ArgWrite, arg_write, Downloader, request, Response, status_ok, RequestError


function curl_download(
        url      :: AbstractString,
        output   :: Union{ArgWrite, Nothing} = nothing;
        headers  :: Union{AbstractVector, AbstractDict} = Pair{String, String}[],
        progress :: Union{Function, Nothing} = nothing,
        verbose  :: Bool = false)

    options = ["-L"]  # Follow redirections

    !verbose && push!(options, "-s", "-S")  # silent mode, but print errors if any
    verbose  && push!(options, "-v")  # Verbose mode

    # Pass the headers to curl
    for (header, content) in headers
        push!(options, "-H", header * ":" * content)
    end

    # Simple implementation: only file names are supported
    if !isa(output, AbstractString)
        error("Cannot handle this output file type: ", typeof(output), " as ", output)
    end

    push!(options, "-o", output)
    curl_cmd = `curl $options $url`

    if verbose
        println("Curl cmd: ", curl_cmd)
    end

    # Run the command, if it fails, print it and rethrow the error
    try
        run(curl_cmd)
    catch
        !verbose && println("Curl cmd failed: ", curl_cmd)
        rethrow()
    end

    return output
end


function Downloads.download(
    url        :: AbstractString,
    output     :: Union{ArgWrite, Nothing} = nothing;
    method     :: Union{AbstractString, Nothing} = nothing,
    headers    :: Union{AbstractVector, AbstractDict} = Pair{String,String}[],
    timeout    :: Real = Inf,
    progress   :: Union{Function, Nothing} = nothing,
    verbose    :: Bool = false,
    downloader :: Union{Downloader, Nothing} = nothing,
) :: ArgWrite
    return curl_download(url, output; headers, progress, verbose)
end

Here we just make a call to the curl via the command line and let it read the ~/.curlrc config file for proper proxy configuration.

Together with JULIA_PKG_USE_CLI_GIT=true, I could download and install CUDA.jl, AMDGPU.jl, LoopVectorization.jl… without any problems.