Bad cert when accessing the General registry

Tooling Package Management
1

I just started getting errors about untrusted SSL cert when accessing the General registry.

If I try to update General:

(@v1.10) pkg> registry update General
    Updating registry at `~/.julia/registries/General.toml`
┌ Error: Some registries failed to update:
│     — /Users/dmatz/.julia/registries/General.toml — failed to download from https://pkg.julialang.org/registry/23338594-aafe-5451-b93e-139f81909106/89c6967ae14017d23abb5eda0c65a7c263ca9aac. Exception: RequestError: HTTP/2 301 (SSL certificate problem: Invalid certificate chain) while requesting https://pkg.julialang.org/registry/23338594-aafe-5451-b93e-139f81909106/89c6967ae14017d23abb5eda0c65a7c263ca9aac
└ @ Pkg.Registry ~/.julia/juliaup/julia-1.10.2+0.aarch64.apple.darwin14/share/julia/stdlib/v1.10/Pkg/src/Registry/Registry.jl:510

Another type of error when my CI system tries to get General in a fresh depot:

$ julia -e 'using Pkg; Pkg.Registry.add("General")'
ERROR: could not download https://pkg.julialang.org/registry/23338594-aafe-5451-b93e-139f81909106/89c6967ae14017d23abb5eda0c65a7c263ca9aac
Stacktrace:
  [1] pkgerror(msg::String)
    @ Pkg.Types /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Pkg/src/Types.jl:55
  [2] (::Pkg.Types.var"#82#85"{Pkg.Types.Context, String, Pkg.Types.RegistrySpec})(tmp::String)
    @ Pkg.Types /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Pkg/src/Types.jl:1035
  [3] mktempdir(fn::Pkg.Types.var"#82#85"{Pkg.Types.Context, String, Pkg.Types.RegistrySpec}, parent::String; prefix::String)
    @ Base.Filesystem ./file.jl:729
  [4] mktempdir (repeats 2 times)
    @ ./file.jl:727 [inlined]
  [5] clone_or_cp_registries(ctx::Pkg.Types.Context, regs::Vector{Pkg.Types.RegistrySpec}, depot::String)
    @ Pkg.Types /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Pkg/src/Types.jl:1027
  [6] clone_or_cp_registries
    @ /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Pkg/src/Types.jl:1020 [inlined]
  [7] #add#7
    @ /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Pkg/src/Registry.jl:33 [inlined]
  [8] add
    @ /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Pkg/src/Registry.jl:29 [inlined]
  [9] #add#5
    @ /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Pkg/src/Registry.jl:26 [inlined]
 [10] add
    @ /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Pkg/src/Registry.jl:26 [inlined]
 [11] add(regs::Vector{String}; kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ Pkg.Registry /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Pkg/src/Registry.jl:25
 [12] add
    @ /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Pkg/src/Registry.jl:25 [inlined]
 [13] #add#1
    @ /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Pkg/src/Registry.jl:24 [inlined]
 [14] add(reg::String)
    @ Pkg.Registry /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Pkg/src/Registry.jl:24
 [15] top-level scope
    @ none:1
caused by: HTTP/2 301 (Cert verify failed: BADCERT_NOT_TRUSTED) while requesting https://pkg.julialang.org/registry/23338594-aafe-5451-b93e-139f81909106/89c6967ae14017d23abb5eda0c65a7c263ca9aac

Is anyone else seeing this?

2

what Linux distribution (and version) are you on?

3

The top example is on my Mac, and the bottom example is a CI job running on our Linux system, which is CentOS 7.

4

Ha. At CERN we have this problem as well, where the stock COS7 with Julia (which is official binary but the our software environment would overload a bunch of things) would give you cert error, but only if you’re not using the official binary from juliaup.

As for your local macOS laptop, I’m not sure, I just tried it now and it works fine.

5

Some co-workers are also seeing the issue, but only when on our company network. Must be an issue on our side.

6

Is there a proxy/firewall? Those will sometimes MITM your connections. In such cases there’s usually some company-specific certificate authority cert that needs to be added to allow the transparent proxy to perform this MITM. It will read all your traffic in such a case, even over HTTPS connections.

3 Likes
7

5 posts were split to a new topic: Unexpected end of data when updating registries with LocalPackageServer

12

If I use my browser to navigate to your us-west URL, it tries to download the registry. If I navigate to us-east, it seems to forward to “storage.julialang.net”, and my browser complains about an invalid certificate.

If I force Julia to use the us-west server, things are working great:

$ JULIA_PKG_SERVER=https://us-west.pkg.julialang.org julia

(To be clear, my original issue was unrelated to HTTP.jl.)

13

The root certificate for https://storage.julialang.net (which https://us-east.pkg.julialang.org redirects to) is ISRG Root X2 and the root certificate for https://us-west.pkg.julialang.org (and https://us-east.pkg.julialang.org too) is ISRG Root X1. Is it possible that the former isn’t trusted (yet) by your system?

See https://letsencrypt.org/certificates/ for some more details. In particular they list a couple of test pages where you can verify the theory above. For example, https://valid-isrgrootx1.letsencrypt.org/ present a valid certificate signed by ISRG Root X1 and https://valid-isrgrootx2.letsencrypt.org/ a valid cert signed by ISRG Root X2.

3 Likes
14

Thanks for that detail, I’ll pass that along. I’ve opened a ticket with my company’s IT department. They must have changed something this week!

2 Likes
15

Not necessarily a change on your end since the redirection to https://storage.julialang.net was deployed yesterday. Regardless, all certs are valid as far as I can tell.

1 Like
16

You can locally try to curl the two letsencrypt URLs that Fredrik posted and check if the https://valid-isrgrootx2.letsencrypt.org/ one doesn’t work. In that case that would confirm the issue.

17

I tried both https://valid-isrgrootx1.letsencrypt.org/ and https://valid-isrgrootx2.letsencrypt.org/ in both my browser and with curl, and everything worked fine. But https://us-east.pkg.julialang.org/registry/23338594-aafe-5451-b93e-139f81909106/6e59100195a18329cf56e7a7b76c065fa25d3a5c continues to give me issues.

18

Did you mean .org? Because I get redirected to https://storage.julialang.net.

19

I meant .net, sorry. Updated.

1 Like
20

My issue bounced through the bureaucracy for a week or so, but it finally made it to the network team. The issue appears to be resolved. Unfortunately, they didn’t leave any notes about what they did, so I’m not sure if it was the root cert or something else.

Thank you all again for help on this!

1 Like