Julia 1.6 other libcurl download issues

Unless you are seeing the same certificate revocation check error as above, this is unlikely to be the same issue. If you want trouble shooting help there’s a couple of things that would be helpful: start a new topic to discuss and include the full error message.

I’m running into a similar issue. I tried setting the environmental variable ENV["JULIA_SSL_NO_VERIFY_HOSTS"] = "github.com" but that didn’t work I keep getting curl_easy_setop: 2/43 errors.

(@v1.6) pkg > up 
 ┌ Error: curl_easy_setopt: 2
└ @ Downloads.Curl /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Downloads/src/Curl/utils.jl:36
┌ Error: curl_easy_setopt: 2
└ @ Downloads.Curl /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Downloads/src/Curl/utils.jl:36
┌ Error: curl_easy_setopt: 2
└ @ Downloads.Curl /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Downloads/src/Curl/utils.jl:36
┌ Error: curl_easy_setopt: 2
└ @ Downloads.Curl /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Downloads/src/Curl/utils.jl:36
┌ Error: curl_easy_setopt: 2
└ @ Downloads.Curl /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Downloads/src/Curl/utils.jl:36
┌ Error: curl_easy_getinfo: 43
└ @ Downloads.Curl /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Downloads/src/Curl/utils.jl:36
┌ Error: curl_easy_setopt: 2
└ @ Downloads.Curl /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Downloads/src/Curl/utils.jl:36
┌ Error: curl_easy_getinfo: 43
└ @ Downloads.Curl /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Downloads/src/Curl/utils.jl:36
┌ Error: curl_easy_setopt: 2
└ @ Downloads.Curl /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Downloads/src/Curl/utils.jl:36
┌ Error: curl_easy_getinfo: 43
└ @ Downloads.Curl /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Downloads/src/Curl/utils.jl:36
┌ Error: curl_easy_getinfo: 43
└ @ Downloads.Curl /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Downloads/src/Curl/utils.jl:36
┌ Error: curl_easy_setopt: 2
└ @ Downloads.Curl /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Downloads/src/Curl/utils.jl:36
┌ Error: curl_easy_setopt: 2
└ @ Downloads.Curl /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Downloads/src/Curl/utils.jl:36
┌ Error: curl_easy_getinfo: 43
└ @ Downloads.Curl /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Downloads/src/Curl/utils.jl:36
┌ Error: curl_easy_setopt: 2
└ @ Downloads.Curl /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Downloads/src/Curl/utils.jl:36
┌ Error: curl_easy_getinfo: 43
└ @ Downloads.Curl /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Downloads/src/Curl/utils.jl:36
┌ Error: curl_easy_getinfo: 43
└ @ Downloads.Curl /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Downloads/src/Curl/utils.jl:36
┌ Error: curl_easy_getinfo: 43
└ @ Downloads.Curl /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Downloads/src/Curl/utils.jl:36
┌ Error: curl_easy_getinfo: 43
└ @ Downloads.Curl /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Downloads/src/Curl/utils.jl:36
┌ Error: curl_easy_getinfo: 43
└ @ Downloads.Curl /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Downloads/src/Curl/utils.jl:36

Also unlikely to be the same issue. My guess is that you are using a system libcurl and it’s old.

Yeah thats probably right. Trying to get IT to update the cluster is an impossible task. Is there anyway to use a different libcurl?

The official Julia binaries ship with a correct version of libcurl and building it from source also downloads and builds the correct version by default. So one has to make the effort to build Julia against the old, broken system libcurl. I would recommend asking the sysadmin to either upgrade libcurl to something recent or rebuild Julia with its own libcurl. You can also point out that having an old libcurl is a security risk and that especially network code should be kept current.

2 Likes

Dear Dr. Karpinski:

I do not want to take time from the Discourse group. There are some cloud computing sites sites that provide a Julia environment, either free or at nominal cost. I will make use of those. Perhaps I will come back to Julia at some future time. Thank you.

Yours,

NeophyteDave

As similar problems sporadically but regularily pop up here, it would be good if we could find the common cause to this, to be able to solve it. Perhaps there are more than one reasons for this, not always equal to everyone where it happens.
What it needs is someone (or more) who suffers from this AND is willing to put some effort into it to resolve it. With the information we have now, it is very difficult to find a solution. The amount of information I see here on discourse is very little.

A major issue with diagnosing these problems is that Pkg “swallows” a lot of errors and just moves on to try the next possible way of getting something. Then at the end you just get a vague “couldn’t fetch this thing” error. This is extremely annoying from a developer perspective. There’s been some effort lately to fix it. It’s possible that @neophytedave is having the same problem as is reported in the other thread, but based on what he’s posted, it’s impossible to tell. There was test code to try posted there to help diagonse the specific problem but that has not been run here, so who knows? @lstagner’s problem is definitely not the same.

1 Like

Some kind of debug mode the user can switch on, try again, and detailed log metric is written to some file the user could provide.
This would be nice. Or something similar.
Still it needs a user who is willing to help.

Dear Mr. Oheil:

I would welcome some sort of debug mode. I really would like to be able to use Julia.

Neophytedave

Dear @neophytedave

Could you try the following commands and report output here?

versioninfo()
using Libdl
libcurl = "libcurl-4.dll"
libcurl_handle = dlopen(libcurl)
libcurl_path = dlpath(libcurl_handle)

This is to identify which curl library is loaded and which Julia system you are running.
Thank you.

Perhaps hope does spring (eventually).

The following (up to “-------------------------------“ below) is output from my latest attempt to add plots. After that is output produced by Mr. Oheil’s request that I enter certain commands at the REPL.

(@v1.6) pkg> add Plots
    Updating registry at `C:\Users\vhaglasmithdi\.julia\registries\General`
    Updating git-repo `https://github.com/JuliaRegistries/General.git`
   Resolving package versions...
  Downloaded artifact: GR
  Downloaded artifact: GR
ERROR: Unable to automatically install 'GR' from 'C:\Users\vhaglasmithdi\.julia\packages\GR_jll\lXOy4\Artifacts.toml'
Stacktrace:
  [1] error(s::String)
    @ Base .\error.jl:33
  [2] ensure_artifact_installed(name::String, meta::Dict{String, Any}, artifacts_toml::String; platform::Base.BinaryPlatforms.Platform, verbose::Bool, quiet_download::Bool)
    @ Pkg.Artifacts C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\Artifacts.jl:436
  [3] ensure_all_artifacts_installed(artifacts_toml::String; platform::Base.BinaryPlatforms.Platform, pkg_uuid::Nothing, include_lazy::Bool, verbose::Bool, quiet_download::Bool)
    @ Pkg.Artifacts C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\Artifacts.jl:501
  [4] download_artifacts(ctx::Pkg.Types.Context, pkg_roots::Vector{String}; platform::Base.BinaryPlatforms.Platform, verbose::Bool)
    @ Pkg.Operations C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\Operations.jl:702
  [5] download_artifacts(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}; platform::Base.BinaryPlatforms.Platform, julia_version::VersionNumber, verbose::Bool)
    @ Pkg.Operations C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\Operations.jl:680
  [6] add(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}, new_git::Vector{Base.UUID}; preserve::Pkg.Types.PreserveLevel, platform::Base.BinaryPlatforms.Platform)
    @ Pkg.Operations C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\Operations.jl:1235
  [7] add(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}; preserve::Pkg.Types.PreserveLevel, platform::Base.BinaryPlatforms.Platform, kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ Pkg.API C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\API.jl:197
  [8] add
    @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\API.jl:148 [inlined]
  [9] add(pkgs::Vector{Pkg.Types.PackageSpec}; kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ Pkg.API C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\API.jl:73
 [10] add(pkgs::Vector{Pkg.Types.PackageSpec})
    @ Pkg.API C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\API.jl:72
 [11] do_cmd!(command::Pkg.REPLMode.Command, repl::REPL.LineEditREPL)
    @ Pkg.REPLMode C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\REPLMode\REPLMode.jl:408
 [12] do_cmd(repl::REPL.LineEditREPL, input::String; do_rethrow::Bool)
    @ Pkg.REPLMode C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\REPLMode\REPLMode.jl:386
 [13] do_cmd
    @ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\REPLMode\REPLMode.jl:377 [inlined]
 [14] (::Pkg.REPLMode.var"#24#27"{REPL.LineEditREPL, REPL.LineEdit.Prompt})(s::REPL.LineEdit.MIState, buf::IOBuffer, ok::Bool)
    @ Pkg.REPLMode C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\REPLMode\REPLMode.jl:550
 [15] #invokelatest#2
    @ .\essentials.jl:708 [inlined]
 [16] invokelatest
    @ .\essentials.jl:706 [inlined]
 [17] run_interface(terminal::REPL.Terminals.TextTerminal, m::REPL.LineEdit.ModalInterface, s::REPL.LineEdit.MIState)
    @ REPL.LineEdit C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\REPL\src\LineEdit.jl:2441
 [18] run_frontend(repl::REPL.LineEditREPL, backend::REPL.REPLBackendRef)
    @ REPL C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\REPL\src\REPL.jl:1126
 [19] (::REPL.var"#44#49"{REPL.LineEditREPL, REPL.REPLBackendRef})()
    @ REPL .\task.jl:406

--------------------------------------------------------------------

julia> versioninfo()
Julia Version 1.6.0
Commit f9720dc2eb (2021-03-24 12:55 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-11.0.1 (ORCJIT, skylake)

julia> using Libdl

julia> libcurl = "libcurl-4.dll"
"libcurl-4.dll"

julia> libcurl_handle = dlopen(libcurl)
Ptr{Nothing} @0x0000000060fe0000

julia> libcurl_path = dlpath(libcurl_handle)
"R:\\RES_Research\\davidsfolder\\Julia-1.6.0\\bin\\libcurl-4.dll"
1 Like

Great, now we know, that the right libcurl is loaded.
Next step is to get some verbose information about your download problem.
Please run the following commands:

cd("c:\\Temp")   #or another folder where you can write to
using Downloads
url = "https://github.com/JuliaBinaryWrappers/GR_jll.jl/releases/download/GR-v0.57.2+0/GR.v0.57.2.x86_64-w64-mingw32-cxx11.tar.gz"
open("out.stdout", "w") do sout
    open("out.stderr", "w") do serr
        redirect_stdout(sout) do
            redirect_stderr(serr) do
                Downloads.download(url;verbose=true)
            end
        end
    end
end

Copy&Paste everything from the REPL here and:
This produces the important file out.stderr with all information while downloading the GR artefact.
Copy&Paste all the content of out.stderr here.

Hopefully we get some hints whats going wrong. See you in 9h, Good Night.

julia> cd(“C:\Users\vhaglasmithdi\temp”)

julia> using Downloads
julia> url = “https://github.com/JuliaBinaryWrappers/GR_jll.jl/releases/download/GR-v0.57.2+0/GR.v0.57.2.x86_64-w64-mingw32-cxx11.tar.gz
https://github.com/JuliaBinaryWrappers/GR_jll.jl/releases/download/GR-v0.57.2+0/GR.v0.57.2.x86_64-w64-mingw32-cxx11.tar.gz

julia> open(“out.stdout”, “w”) do sout
open(“out.stderr”, “w”) do serr
redirect_stdout(sout) do
redirect_stderr(serr) do
Downloads.download(url;verbose=true)
end
end
end
end
ERROR: HTTP/1.1 302 Found (schannel: next InitializeSecurityContext failed: Unknown error (0x80092012) - The revocation function was unable to check revocation for the certificate.) while requesting https://github.com/JuliaBinaryWrappers/GR_jll.jl/releases/download/GR-v0.57.2+0/GR.v0.57.2.x86_64-w64-mingw32-cxx11.tar.gz
Stacktrace:
[1] (::Downloads.var"#9#18"{IOStream, Base.DevNull, Nothing, Vector{Pair{String, String}}, Float64, Nothing, Bool, Bool, String, Int64, Bool, Bool})(easy::Downloads.Curl.Easy)
@ Downloads C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Downloads\src\Downloads.jl:356
[2] with_handle(f::Downloads.var"#9#18"{IOStream, Base.DevNull, Nothing, Vector{Pair{String, String}}, Float64, Nothing, Bool, Bool, String, Int64, Bool, Bool}, handle::Downloads.Curl.Easy)
@ Downloads.Curl C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Downloads\src\Curl\Curl.jl:60
[3] #8
@ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Downloads\src\Downloads.jl:298 [inlined]
[4] arg_write(f::Downloads.var"#8#17"{Base.DevNull, Nothing, Vector{Pair{String, String}}, Float64, Nothing, Bool, Bool, String, Int64, Bool, Bool}, arg::IOStream)
@ ArgTools C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\ArgTools\src\ArgTools.jl:112
[5] #7
@ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Downloads\src\Downloads.jl:297 [inlined]
[6] arg_read
@ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\ArgTools\src\ArgTools.jl:61 [inlined]
[7] request(url::String; input::Nothing, output::IOStream, method::Nothing, headers::Vector{Pair{String, String}}, timeout::Float64, progress::Nothing, verbose::Bool, throw::Bool, downloader::Nothing)
@ Downloads C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Downloads\src\Downloads.jl:296
[8] (::Downloads.var"#3#4"{Nothing, Vector{Pair{String, String}}, Float64, Nothing, Bool, Nothing, String})(output::IOStream)
@ Downloads C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Downloads\src\Downloads.jl:209
[9] arg_write(f::Downloads.var"#3#4"{Nothing, Vector{Pair{String, String}}, Float64, Nothing, Bool, Nothing, String}, arg::Nothing)
@ ArgTools C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\ArgTools\src\ArgTools.jl:101
[10] #download#2
@ C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Downloads\src\Downloads.jl:208 [inlined]
[11] (::var"#12#16")()
@ Main .\REPL[29]:5
[12] redirect_stderr(f::var"#12#16", stream::IOStream)
@ Base .\stream.jl:1224
[13] #11
@ .\REPL[29]:4 [inlined]
[14] redirect_stdout(f::var"#11#15"{IOStream}, stream::IOStream)
@ Base .\stream.jl:1224
[15] #10
@ .\REPL[29]:3 [inlined]
[16] open(::var"#10#14"{IOStream}, ::String, ::Vararg{String, N} where N; kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
@ Base .\io.jl:330
[17] open
@ .\io.jl:328 [inlined]
[18] #9
@ .\REPL[29]:2 [inlined]
[19] open(::var"#9#13", ::String, ::Vararg{String, N} where N; kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
@ Base .\io.jl:330
[20] open(::Function, ::String, ::String)
@ Base .\io.jl:328
[21] top-level scope
@ REPL[29]:1

julia>

Your problem could surely be elsewhere, but it seems that many GR issues can be solved by the following. Maybe worth a try.

ENV["GRDIR"]=""
] build GR

That was a no go.

This is the problem here.
It can be caused for example by aggressive Anti Virus Tools which reroot network traffic over a local man-in-the-middle proxy to be able to scan the network traffic even if encrypted. Or by not so well configured corporate proxys.

There is a Curl option --ssl-no-revoke for cases like this, but I don’t know how to inject this option into the download system of Julia.
It is something like:

using Pkg, Downloads
easy=Downloads.Curl.Easy()
Downloads.Curl.setopt(easy,Downloads.Curl.CURLOPT_PROXY_SSL_OPTIONS,Downloads.Curl.CURLSSLOPT_NO_REVOKE)
Pkg.add("Plots")

It’s worth a try @neophytedave but I don’t expect this to work.

Perhaps @StefanKarpinski can give a hint if it is possible to set this option before using add or download ?

Or is it possible to set curl options in a user .curl file?

Create a file .curlrc in your home folder (C:\Users\vhaglasmithdi)

add a line with
ssl-no-revoke
in the file.

Answer is No for libcurl, see below.

As already pointed out by Stephan There, libcurl does not answer to .curlrc files. Also you can look at the thread for details on what we already tried in this schanel issue.

Setting Julia_ssl_no_verify AND giving http_proxy and https_proxy according to your corporate proxy should solve the issue, as reported on the accompanying GitHub issue that we files the other day There.

The environnement variable to avoid verifying hosts + the http & https credentials of your proxy should be enough to get rid of this schanel issue.