Hi All,
I am facing problems downloading artifacts on a linux remote server I connect to via SSH. This is an issue that I have been facing only in the last days and I am using this server daily since months now.
This is my versioninfo() output:
julia> versioninfo()
Julia Version 1.11.4
Commit 8561cc3d68d (2025-03-10 11:36 UTC)
Build Info:
Official https://julialang.org/ release
Platform Info:
OS: Linux (x86_64-linux-gnu)
CPU: 24 × Intel(R) Xeon(R) Platinum 8462Y+
WORD_SIZE: 64
LLVM: libLLVM-16.0.6 (ORCJIT, haswell)
Threads: 24 default, 0 interactive, 12 GC (on 24 virtual cores)
Environment:
JULIA_PKG_USE_CLI_GIT = true
I am having issues downloading the General registry (get the warning below) but this has so far not caused specific error per se.
The real error happens when trying to download artifacts, and appeared first when trying to download XML2_jll package via indirect dependency, triggering a similar error for both the pkgserver and the github fallback:
I tried to pinpoint this issue, and I basically face the same problem when trying to directly download the tar.gz via Downloads.download, with the following output when doing verbose=true
julia> Downloads.download("https://github.com/JuliaBinaryWrappers/XML2_jll.jl/releases/download/XML2-v2.14.1+0/XML2.v2.14.1.x86_64-linux-gnu.tar.gz"; verbose = true)
* Couldn't find host github.com in the .netrc file; using defaults
* Host github.com:443 was resolved.
* IPv6: (none)
* IPv4: 140.82.121.3
* Trying 140.82.121.3:443...
* Connected to github.com (140.82.121.3) port 443
* mbedTLS: Connecting to github.com:443
* mbedTLS: Set min SSL version to TLS 1.0
* ALPN: curl offers h2,http/1.1
* Recv failure: Connection reset by peer
* ssl_handshake returned - mbedTLS: (-0x0001) ERROR - Generic error
* Closing connection
ERROR: RequestError: Recv failure: Connection reset by peer while requesting https://github.com/JuliaBinaryWrappers/XML2_jll.jl/releases/download/XML2-v2.14.1+0/XML2.v2.14.1.x86_64-linux-gnu.tar.gz
Strangely enough, when trying to directly download the same file on the shell using
curl -O -L https://github.com/JuliaBinaryWrappers/XML2_jll.jl/releases/download/XML2-v2.14.1+0/XML2.v2.14.1.x86_64-linux-gnu.tar.gz
The downloads succeeds without problems.
Does anybody have any idea on how I could further investigate this issue or solve it?
1 Like
I renamed the thread as the main issues seems to be really just on Downloads.download throwing an error on most of https link including github/pkg server which end up making Artifact downloads impossible on this remote machine.
I check possible previous threads with issues with Downloads like:
I have also seen that there used to be a way to override the downloader used by julia by setting the BINARYPROVIDER_DOWNLOAD_ENGINE variable but this does not seem to be applicable anymore in recent julia versions.
I initially though this issue might be temporary on the server I am working on but it’s now multiple days that is going on and it’s making my julia use on this linux host very problematic as a lot of packages now directly or indirectly rely on artifacts.
Is there a way to make julia use the system’s curl somehow for Artifacts (and or the whole Pkg stuff?) I doubt so but maybe worth asking.
Also shamelessly pinging @StefanKarpinski directly (sorry for the bother) as I see you were often answering questions with Downloads.download issues.
Do you get these problems with a completely fresh install? Specifically I am thinking of (temporarily) renaming your ~/.julia directory, reinstalling julia (with juliaup) and unsetting JULIA_PKG_USE_CLI_GIT and trying to install something from General registry. If that doesn’t work then try installing a package with the explicit URL, e.g. ] add https://github.com/JuliaLang/Example.jl. This will also test whether your problem will be solved by upgrading to 1.11.5
Hi, Thanks for the suggestions but I unofrtunately confirm that removing the depot and going to 1.11.5 (without OS git) does not helps on this
1 Like
Hi @StefanKarpinski,
Thanks a lot for taking the time to reply here. Regarding your questions:
- The server is inside my company’s network and for sure there is a firewall (not sure about transaprent proxy). I don’t know the exact details on the firewall per se. I do have access to a full shell on the server itself (but without admin rights) in case there is something I can run to help clarify this better
- I installed julia with juliaup, so I do believe libcurl should be the one shipped with julia itself. I also never had issues before 5-6 days ago and nothing else changed on my side, so I assume some policy changes on firewall happened
As a side note, I also have a bunch of packages that are hosted on a company-wide gitlab instance hosted on company infrastracture. (Which I assume is subject to same or similar firewall policies).
Also on CI on this private gitlab instance, I do see the same problem on linux runners (but not on windows ones). And in that case I have CI jobs starting from the plain julia docker image.
Let me know if there is any command or futher investigation I might do to help diagnose the issue
A small update on this. I was able to override Pkg’s inner download function (thanks julia’s flexibility for this :D) to use the OS curl if it tries to download from https://pkg.julialang.org/
I do something very cursed (as per code on this gist) which is basically overwriting the function Pkg.PlatformEngines.download to substitute these lines:
into these other lines:
try
if startswith(url, "https://pkg.julialang.org")
# We override the default download command with this cursed function to use the OS's curl for requests to pkg.julialang.org
cmd = `curl -sS -L -o $dest`
for header in headers
key, val = header
push!(cmd.exec, "-H", "$key: $val")
end
push!(cmd.exec, url)
run(cmd)
else
Downloads.download(url, dest; headers, progress)
end
finally
By doing this, I confirm that I can install packages and Artifacts both on the server I connect to via SSH and also within our gitlab CI runners.
It is not pretty but at least allows me (and my colleagues) to keep working normally
I assume this means that firewall is not completely blocking links to pkgserver or github and hopefully it will be possible to find a cleaner solution for this (e.g. passing the correct parameters to libcurl used by Downloads.download)
