Julia 1.3.1 cannot automatically install

Hi!

I moved from version 1.2 to 1.3.1 and get the following error when trying to build anything:

ERROR: Unable to automatically install 'Expat' from 'C:\Users\ramirov\.julia\packages\Expat_jll\YXYuQ\Artifacts.toml'
Stacktrace:
 [1] error(::String) at .\error.jl:33
 [2] #ensure_artifact_installed#42(::Pkg.BinaryPlatforms.Platform, ::Bool, ::typeof(Pkg.Artifacts.ensure_artifact_installed), ::String, ::Dict{String,Any}, ::String) at C:\cygwin\home\ramirov\codigos\julia\usr\share\julia\stdlib\v1.3\Pkg\src\Artifacts.jl:873
 [3] (::Pkg.Artifacts.var"#kw##ensure_artifact_installed")(::NamedTuple{(:platform, :verbose),Tuple{Pkg.BinaryPlatforms.Windows,Bool}}, ::typeof(Pkg.Artifacts.ensure_artifact_installed), ::String, ::Dict{String,Any}, ::String) at .\none:0
 [4] #ensure_all_artifacts_installed#43(::Pkg.BinaryPlatforms.Platform, ::Nothing, ::Bool, ::Bool, ::typeof(Pkg.Artifacts.ensure_all_artifacts_installed), ::String) at C:\cygwin\home\ramirov\codigos\julia\usr\share\julia\stdlib\v1.3\Pkg\src\Artifacts.jl:919
 [5] #ensure_all_artifacts_installed at C:\cygwin\home\ramirov\codigos\julia\usr\share\julia\stdlib\v1.3\Pkg\src\Operations.jl:0 [inlined]
 [6] #download_artifacts#78(::Pkg.BinaryPlatforms.Windows, ::Bool, ::typeof(Pkg.Operations.download_artifacts), ::Array{String,1}) at C:\cygwin\home\ramirov\codigos\julia\usr\share\julia\stdlib\v1.3\Pkg\src\Operations.jl:580
 [7] #download_artifacts at .\none:0 [inlined]
 [8] #download_artifacts#73(::Pkg.BinaryPlatforms.Windows, ::Bool, ::typeof(Pkg.Operations.download_artifacts), ::Array{Pkg.Types.PackageSpec,1}) at C:\cygwin\home\ramirov\codigos\julia\usr\share\julia\stdlib\v1.3\Pkg\src\Operations.jl:570
 [9] (::Pkg.Operations.var"#kw##download_artifacts")(::NamedTuple{(:platform,),Tuple{Pkg.BinaryPlatforms.Windows}}, ::typeof(Pkg.Operations.download_artifacts), ::Array{Pkg.Types.PackageSpec,1}) at .\none:0
 [10] #add#112(::Bool, ::Pkg.BinaryPlatforms.Windows, ::typeof(Pkg.Operations.add), ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}, ::Array{Base.UUID,1}) at C:\cygwin\home\ramirov\codigos\julia\usr\share\julia\stdlib\v1.3\Pkg\src\Operations.jl:1017
 [11] #add at .\none:0 [inlined]
 [12] #add#25(::Bool, ::Pkg.BinaryPlatforms.Windows, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(Pkg.API.add), ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at C:\cygwin\home\ramirov\codigos\julia\usr\share\julia\stdlib\v1.3\Pkg\src\API.jl:102
 [13] add(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at C:\cygwin\home\ramirov\codigos\julia\usr\share\julia\stdlib\v1.3\Pkg\src\API.jl:72
 [14] do_add!(::Dict{Symbol,Any}, ::Array{Pkg.Types.PackageSpec,1}, ::Dict{Symbol,Any}) at C:\cygwin\home\ramirov\codigos\julia\usr\share\julia\stdlib\v1.3\Pkg\src\REPLMode.jl:505
 [15] #invokelatest#1(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(Base.invokelatest), ::Any, ::Any, ::Vararg{Any,N} where N) at .\essentials.jl:709
 [16] invokelatest(::Any, ::Any, ::Vararg{Any,N} where N) at .\essentials.jl:708
 [17] do_cmd!(::Pkg.REPLMode.Command, ::REPL.LineEditREPL) at C:\cygwin\home\ramirov\codigos\julia\usr\share\julia\stdlib\v1.3\Pkg\src\REPLMode.jl:412
 [18] #do_cmd#23(::Bool, ::typeof(Pkg.REPLMode.do_cmd), ::REPL.LineEditREPL, ::String) at C:\cygwin\home\ramirov\codigos\julia\usr\share\julia\stdlib\v1.3\Pkg\src\REPLMode.jl:391
 [19] do_cmd at C:\cygwin\home\ramirov\codigos\julia\usr\share\julia\stdlib\v1.3\Pkg\src\REPLMode.jl:387 [inlined]
 [20] (::Pkg.REPLMode.var"#28#31"{REPL.LineEditREPL,REPL.LineEdit.Prompt})(::REPL.LineEdit.MIState, ::Base.GenericIOBuffer{Array{UInt8,1}}, ::Bool) at C:\cygwin\home\ramirov\codigos\julia\usr\share\julia\stdlib\v1.3\Pkg\src\REPLMode.jl:619
 [21] #invokelatest#1 at .\essentials.jl:709 [inlined]
 [22] invokelatest at .\essentials.jl:708 [inlined]
 [23] run_interface(::REPL.Terminals.TextTerminal, ::REPL.LineEdit.ModalInterface, ::REPL.LineEdit.MIState) at C:\cygwin\home\ramirov\codigos\julia\usr\share\julia\stdlib\v1.3\REPL\src\LineEdit.jl:2306
 [24] run_frontend(::REPL.LineEditREPL, ::REPL.REPLBackendRef) at C:\cygwin\home\ramirov\codigos\julia\usr\share\julia\stdlib\v1.3\REPL\src\REPL.jl:1045
 [25] run_repl(::REPL.AbstractREPL, ::Any) at C:\cygwin\home\ramirov\codigos\julia\usr\share\julia\stdlib\v1.3\REPL\src\REPL.jl:201
 [26] (::Base.var"#770#772"{Bool,Bool,Bool,Bool})(::Module) at .\client.jl:382
 [27] #invokelatest#1 at .\essentials.jl:709 [inlined]
 [28] invokelatest at .\essentials.jl:708 [inlined]
 [29] run_main_repl(::Bool, ::Bool, ::Bool, ::Bool, ::Bool) at .\client.jl:366
 [30] exec_options(::Base.JLOptions) at .\client.jl:304
 [31] _start() at .\client.jl:460

I have deleted everything before moving from 1.2 to 1.3.1 version and used the current stable release in https://julialang.org/downloads/. Also, once I got the error, I tried cloning Julia from its repository and using make, but the result was the same.

I also get the same error but instead of saying Expat it says Arpack or OpenSpecFun.

Any ideas?

Let me show you some information:

$ umask
0022
julia> versioninfo()
Julia Version 1.3.1
Commit 2d5741174c (2019-12-30 21:36 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: Intel(R) Core(TM) i3-4160 CPU @ 3.60GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, haswell)

Maybe @staticfloat can help me…

Thanks a lot!

That error message isn’t so nice. It would be good if it could say more precisely what failed.

Hi @kristoffer.carlsson, thanks for your reply.

In a fresh Julia v1.3.1 installation, when I try to install a package I get this error. For example, for SpecialFunctions I get the same error but instead of ‘Expat’, the error says ‘OpenSpecFun’. For other packages, I get the error for ‘Arpack’.

I believe the issue is related to the files that are listed in the Artifacts.toml file.

I moved to the first release candidate for Julia v1.4.0 and got a better error message, saying something about untrusted certificate. I cannot post the error message because I get that when I am using my company internet configuration at work and I have already left.

So the error seems to be related to SSL verification or something like that.

However, I have not been able to bypass this issue although I set the sslverify flag to false using git config --global ...

I can manually download those files but I do not know where to put them. On the other hand, it would be really uncomfortable to do it manually for all packages.

Any ideas?

Thanks!

@rvignolo what does the following say for you:

using Pkg
Pkg.PlatformEngines.probe_platform_engines!(; verbose=true)
mktempdir() do dir
    url = "https://github.com/JuliaBinaryWrappers/Expat_jll.jl/releases/download/Expat-v2.2.7+0/Expat.v2.2.7.x86_64-w64-mingw32.tar.gz"
    Pkg.PlatformEngines.download(url, joinpath(dir, "test.tar.gz"); verbose=true)
end

Hi @staticfloat! First of all, thank you for your reply.
Firs, versioninfo for completeness:

julia> versioninfo()
Julia Version 1.4.0-rc1.0
Commit b0c33b0cf5 (2020-01-23 17:23 UTC)
Platform Info:
 OS: Windows (x86_64-w64-mingw32)
 CPU: Intel(R) Core(TM) i3-4160 CPU @ 3.60GHz
 WORD_SIZE: 64
 LIBM: libopenlibm
 LLVM: libLLVM-8.0.1 (ORCJIT, haswell)

When running your script, I get the following error:

julia> using Pkg

julia> Pkg.PlatformEngines.probe_platform_engines!(; verbose=true)
[ Info: Probing for download engine...
[ Info: Probing powershell as a possibility...
[ Info:   Probe successful for powershell
[ Info: Found download engine powershell
[ Info: Probing for compression engine...
[ Info: Probing C:\cygwin\home\RamiroV\codigos\julia\usr\bin\..\libexec\7z.exe as a possibility...
[ Info: Probing C:\cygwin\home\RamiroV\codigos\julia\usr\bin\7z.exe as a possibility...
[ Info:   Probe successful for C:\cygwin\home\RamiroV\codigos\julia\usr\bin\7z.exe
[ Info: Found compression engine C:\cygwin\home\RamiroV\codigos\julia\usr\bin\7z.exe
true

julia> mktempdir() do dir
           url = "https://github.com/JuliaBinaryWrappers/Expat_jll.jl/releases/download/Expat-v2.2.7+0/Expat.v2.2.7.x86_64-w64-mingw32.tar.gz"
           Pkg.PlatformEngines.download(url, joinpath(dir, "test.tar.gz"); verbose=true)
       end
Exception setting "SecurityProtocol": "Cannot convert null to type "System.Net.SecurityProtocolType" due to invalid enu
meration values. Specify one of the following enumeration values and try again. The possible enumeration values are "Ss
l3, Tls"."
At line:1 char:35
+ [System.Net.ServicePointManager]:: <<<< SecurityProtocol =
    + CategoryInfo          : InvalidOperation: (:) [], RuntimeException
    + FullyQualifiedErrorId : PropertyAssignmentException

Exception calling "DownloadFile" with "2" argument(s): "The underlying connection was closed: An unexpected error occur
red on a send."
At line:7 char:24
+ $webclient.DownloadFile <<<< ("https://github.com/JuliaBinaryWrappers/Expat_jll.jl/releases/download/Expat-v2.2.7+0/E
xpat.v2.2.7.x86_64-w64-mingw32.tar.gz", "C:\cygwin\tmp\jl_Q7VSjE\test.tar.gz")
    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : DotNetMethodException

ERROR: Could not download https://github.com/JuliaBinaryWrappers/Expat_jll.jl/releases/download/Expat-v2.2.7+0/Expat.v2.2.7.x86_64-w64-mingw32.tar.gz to C:\cygwin\tmp\jl_Q7VSjE\test.tar.gz:
ProcessFailedException(Base.Process[Process(`powershell -NoProfile -Command '[System.Net.ServicePointManager]::SecurityProtocol =
    [System.Net.SecurityProtocolType]::Tls12;
$webclient = (New-Object System.Net.Webclient);
$webclient.UseDefaultCredentials = $true;
$webclient.Proxy.Credentials = $webclient.Credentials;
$webclient.Headers.Add("user-agent", "Pkg.jl (https://github.com/JuliaLang/Pkg.jl)");
$webclient.DownloadFile("https://github.com/JuliaBinaryWrappers/Expat_jll.jl/releases/download/Expat-v2.2.7+0/Expat.v2.2.7.x86_64-w64-mingw32.tar.gz", "C:\cygwin\tmp\jl_Q7VSjE\test.tar.gz")
'`, ProcessExited(1))])
Stacktrace:
 [1] error(::String) at .\error.jl:33
 [2] download(::String, ::String; verbose::Bool, auth_header::Nothing) at C:\cygwin\home\ramirov\codigos\julia\usr\share\julia\stdlib\v1.4\Pkg\src\PlatformEngines.jl:846
 [3] #3 at .\REPL[4]:3 [inlined]
 [4] mktempdir(::var"#3#4", ::String; prefix::String) at .\file.jl:682
 [5] mktempdir(::Function, ::String) at .\file.jl:680 (repeats 2 times)
 [6] top-level scope at REPL[4]:1
caused by [exception 1]
failed process: Process(`powershell -NoProfile -Command '[System.Net.ServicePointManager]::SecurityProtocol =
    [System.Net.SecurityProtocolType]::Tls12;
$webclient = (New-Object System.Net.Webclient);
$webclient.UseDefaultCredentials = $true;
$webclient.Proxy.Credentials = $webclient.Credentials;
$webclient.Headers.Add("user-agent", "Pkg.jl (https://github.com/JuliaLang/Pkg.jl)");
$webclient.DownloadFile("https://github.com/JuliaBinaryWrappers/Expat_jll.jl/releases/download/Expat-v2.2.7+0/Expat.v2.2.7.x86_64-w64-mingw32.tar.gz", "C:\cygwin\tmp\jl_Q7VSjE\test.tar.gz")
'`, ProcessExited(1)) [1]

Stacktrace:
 [1] pipeline_error at .\process.jl:525 [inlined]
 [2] run(::Cmd, ::Tuple{Base.DevNull,Base.TTY,Base.TTY}; wait::Bool) at .\process.jl:440
 [3] run(::Cmd, ::Tuple{Base.DevNull,Base.TTY,Base.TTY}) at .\process.jl:438
 [4] download(::String, ::String; verbose::Bool, auth_header::Nothing) at C:\cygwin\home\ramirov\codigos\julia\usr\share\julia\stdlib\v1.4\Pkg\src\PlatformEngines.jl:841
 [5] #3 at .\REPL[4]:3 [inlined]
 [6] mktempdir(::var"#3#4", ::String; prefix::String) at .\file.jl:682
 [7] mktempdir(::Function, ::String) at .\file.jl:680 (repeats 2 times)
 [8] top-level scope at REPL[4]:100:

It seems to me that there might be two problems:

  1. The first one is my PowerShell version (I mean, there is a problem with Tls12 enum …), which I cannot update. Is there any way I can use another download engine? I have cygwin therefore I have access to some GNU/linux commands.
     PS C:\Users\ramirov> $PSVersionTable.PSVersion
    
     Major  Minor  Build  Revision
    -----  -----  -----  --------
    2      0      -1     -1
    
  2. Once item 1. is resolved, I would probably get an error related to SSL verification, which I have been able to solve every time by setting ssl verification flags to false.

FYI: I have been using Julia version 1.2 in this machine. I used to get errors related to downloads before, but I could solve them by manually downloading those files and putting them in the corresponding place. I think that if I use a different download engine that replaces PowerShell many issues will be solved. Updating PowerShell requires that I raise a ticket in my company and try to explain to someone why would I need to update PowerShell. That is a really tedious task.

Thanks in advance!

Ramiro.

Are you behind a corporate proxy by any chance? It is possible that this issue affects you.

(As you also mentioned this in your other post), please, please do not do this anymore. You are putting your company and its customers’ data at risk and they will hold you accountable if anything happens. Your computer is about to execute software that you download over that connection, and you don’t want it to be replaced by malicious content.

No, I am not behind any proxy. I am able to download data or clone repositories from GitHub. In the issue you mention, they were not able to clone https://github.com/JuliaRegistries/General.git, but I am. My issue is related to the data files detailed in Artifacts.toml files, which are downloaded using PowerShell commands as I understand.

Yes, we are aware of this. There is no company or customers data at risk since there is no data at all :smile: . Although, thank you for your suggestion.

I believe the problem is with PowerShell version. Any thoughts?

So the fundamental issue is that you have a powershell that can’t talk with the internet at large because it doesn’t support TLS v1.2. To fix this, you should read the Windows platform instructions and install the TLS easy fix, then update your powershell: https://julialang.org/downloads/platform/

If you can’t do that, you can try installing curl, then setting the environment variable BINARYPROVIDER_DOWNLOAD_ENGINE to the value curl. Do that, then run the commands I gave you above again.

1 Like

@staticfloat thanks for your reply!

Your suggestion solves the first issue! Then, I get the secondary issue, which I suspected. Any ideas how to set ssl verification flag to false for curl from within Julia?

julia> ENV["BINARYPROVIDER_DOWNLOAD_ENGINE"]
"curl"

julia> using Pkg

julia> Pkg.PlatformEngines.probe_platform_engines!(; verbose=true)
[ Info: Probing for download engine...
[ Info: Probing curl as a possibility...
[ Info:   Probe successful for curl
[ Info: Found download engine curl
[ Info: Probing for compression engine...
[ Info: Probing C:\cygwin\home\RamiroV\codigos\julia\usr\bin\..\libexec\7z.exe as a possibility...
[ Info: Probing C:\cygwin\home\RamiroV\codigos\julia\usr\bin\7z.exe as a possibility...
[ Info:   Probe successful for C:\cygwin\home\RamiroV\codigos\julia\usr\bin\7z.exe
[ Info: Found compression engine C:\cygwin\home\RamiroV\codigos\julia\usr\bin\7z.exe
true

julia> mktempdir() do dir
           url = "https://github.com/JuliaBinaryWrappers/Expat_jll.jl/releases/download/Expat-v2.2.7+0/Expat.v2.2.7.x86_64-w64-mingw32.tar.gz"
           Pkg.PlatformEngines.download(url, joinpath(dir, "test.tar.gz"); verbose=true)
       end
-=O=-                                                   #    #    #   #
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
ERROR: Could not download https://github.com/JuliaBinaryWrappers/Expat_jll.jl/releases/download/Expat-v2.2.7+0/Expat.v2.2.7.x86_64-w64-mingw32.tar.gz to C:\cygwin\tmp\jl_ZmLijP\test.tar.gz:
ProcessFailedException(Base.Process[Process(`curl -C - '-#' -f -o 'C:\cygwin\tmp\jl_ZmLijP\test.tar.gz' -L https://github.com/JuliaBinaryWrappers/Expat_jll.jl/releases/download/Expat-v2.2.7+0/Expat.v2.2.7.x86_64-w64-mingw32.tar.gz`, ProcessExited(60))])
Stacktrace:
 [1] error(::String) at .\error.jl:33
 [2] download(::String, ::String; verbose::Bool, auth_header::Nothing) at C:\cygwin\home\ramirov\codigos\julia\usr\share\julia\stdlib\v1.4\Pkg\src\PlatformEngines.jl:846
 [3] #3 at .\REPL[4]:3 [inlined]
 [4] mktempdir(::var"#3#4", ::String; prefix::String) at .\file.jl:682
 [5] mktempdir(::Function, ::String) at .\file.jl:680 (repeats 2 times)
 [6] top-level scope at REPL[4]:1
caused by [exception 1]
failed process: Process(`curl -C - '-#' -f -o 'C:\cygwin\tmp\jl_ZmLijP\test.tar.gz' -L https://github.com/JuliaBinaryWrappers/Expat_jll.jl/releases/download/Expat-v2.2.7+0/Expat.v2.2.7.x86_64-w64-mingw32.tar.gz`, ProcessExited(60)) [60]

Stacktrace:
 [1] pipeline_error at .\process.jl:525 [inlined]
 [2] run(::Cmd, ::Tuple{Base.DevNull,Base.TTY,Base.TTY}; wait::Bool) at .\process.jl:440
 [3] run(::Cmd, ::Tuple{Base.DevNull,Base.TTY,Base.TTY}) at .\process.jl:438
 [4] download(::String, ::String; verbose::Bool, auth_header::Nothing) at C:\cygwin\home\ramirov\codigos\julia\usr\share\julia\stdlib\v1.4\Pkg\src\PlatformEngines.jl:841
 [5] #3 at .\REPL[4]:3 [inlined]
 [6] mktempdir(::var"#3#4", ::String; prefix::String) at .\file.jl:682
 [7] mktempdir(::Function, ::String) at .\file.jl:680 (repeats 2 times)
 [8] top-level scope at REPL[4]:1

Thanks!

@staticfloat

For example, using this command directly from the cygwin console works:

$ curl -k -C - '-#' -f -o 'test.tar.gz' -L https://github.com/JuliaBinaryWrappers/Expat_jll.jl/releases/download/Expat-v2.2.7+0/Expat.v2.2.7.x86_64-w64-mingw32.tar.gz
######################################################################################################################################################################################### 100.0%

Where I only added the flag -k from what Pkg does.

In this context, is there any way I can add this flag to Pkg instruction? Or should I change the source code and recompile?

If this is the case, should I use the Julia repository and change code in PlatformEngine.jl and then make?

Thanks!

EDIT: I created the .curlrc file and added the option! That works. Perfect! thanks!

While I’m glad to know that it can work, for anyone else reading this thread, I really, strongly discourage use of the -k flag with curl, or anything else that disables SSL verification. If you do this for everything, a malicious party can install arbitrary code on your machine without your knowledge. So if at all possible, please do not do this and instead figure out what’s wrong with the SSL certificates on your machine and update them.

2 Likes

Yes of course.

I really appreciate your help! Thank you very much and have a great day

For reference for others troubleshooting this, on my machine, I was getting this error due to https://github.com/JuliaPackaging/BinaryProvider.jl/issues/127. I think the workaround is still the same though.