Issue with installing JuliaUp on compute cluster

New to Julia
1

I am new to installing Julia on systems, but recently I needed to install it on our compute cluster. I tried using the standard “curl” approach given on the Julia installation webpage. I get the following error, which I don’t quite understand:

Now installing Juliaup
Error: Failed to download from url `https://julialang-s3.julialang.org/juliaup/bin/juliaup-1.17.4-x86_64-unknown-linux-musl.tar.gz`.

Caused by:
    0: error sending request for url (https://julialang-s3.julialang.org/juliaup/bin/juliaup-1.17.4-x86_64-unknown-linux-musl.tar.gz)
    1: client error (Connect)
    2: invalid peer certificate: UnknownIssuer

I am able to wget the tarball from the cluster node, so I don’t believe it is a web access issue. Is there any way to get more information about what is breaking here? Are there any suggestions on how to debug this crash?

2

You can try downloading using wget instead of curl.

  1. download the install script
  2. modify line 497 to make it like line 499
  3. run the script
3

Thank you for your help! I just tried out your suggested approach. Running the script forcing it to use wget resulted in the same error message. I commented out the “rm” for the installer binary (juliainstaller) in the /tmp directory , and verified it did download it correctly. So it doesn’t seem to be an issue with the install script, but within the actual installer.

4

Are you behind a firewall that restricts outgoing traffic and/or uses a proxy?

If you go to a web browser (on one of the machines on the same network), and you type http://julialang-s3.julialang.org/ in the address bar, what do you get?

5

I don’t believe our HPC cluster has very restrictive firewall. I was able to download the tarball manually using wget, so I was able to access https://julialang-s3.julialang.org/juliaup/bin/juliaup-1.17.4-x86_64-unknown-linux-musl.tar.gz.

Our nodes do not have a browser installed (they are geared towards being non-interactive), but I can ping the URL http://julialang-s3.julialang.org successfully. Also, when I run a curl -is https://julialang-s3.julialang.org/ I get the following stdout:

curl -is http://julialang-s3.julialang.org
HTTP/1.1 301 Moved Permanently
Connection: close
Content-Length: 0
Server: Varnish
Retry-After: 0
Location: https://julialang-s3.julialang.org/
Accept-Ranges: bytes
Date: Wed, 18 Sep 2024 20:15:47 GMT
Via: 1.1 varnish
X-Served-By: cache-lga21930-LGA
X-Cache: HIT
X-Cache-Hits: 0
X-Timer: S1726690547.205015,VS0,VE0

Which I think means it can see the URL and does not get something like a 404. Sorry I can’t do the more straightforward browser test.

6

How about

curl -is https://julialang-s3.julialang.org

(with https)
you should get a really long xml.

7

If I tell curl to follow the redirection (i.e., curl -isL https://julialang-s3.julialang.org), I end up getting a really long xml. Am I supposed to get that xml directly from https://julialang-s3.julialang.org?

8

The point was to make sure you can access the domain, which you already demonstrated earlier.
At this stage, I would install julia manually.

Edit: Your issue could be related to this.

9

To me this looks as if the https certificate that is presented to the Juliaup installer is not on the list of trusted certs. I think that could happen either 1) because there is some proxy sitting between the computer from which you are downloading and the official Julia servers that is doing something funky, or 2) the server nodes doesn’t have an up-to-date list of trusted certificates.

10

I appreciate everyone’s help. As @dqeeq suggested, I ended up installing juliaup manually using the instructions given in the following thread (Fail to install with curl · Issue #934 · JuliaLang/juliaup · GitHub). That seemed to work, although juliaup update fails with a similar error as before:

> juliaup update
Error: Failed to update versions db.

Caused by:
    0: Failed to download current version db version.
    1: Failed to download from url `https://julialang-s3.julialang.org/juliaup/RELEASECHANNELDBVERSION`.
    2: error sending request for url (https://julialang-s3.julialang.org/juliaup/RELEASECHANNELDBVERSION)
    3: client error (Connect)
    4: invalid peer certificate: UnknownIssuer

Without better knowledge about how our cluster handles certificates, I might have to live with this for now.

11

Yeah, you won’t really be able to use Juliaup if it can’t verify that the downloads it is trying to make are not tampered with… So this is kind of by design…

1 Like
12

This may be a case where it makes more sense for you to just download Julia manually (instead of using Juliaup).

1 Like
13

I’m one of the admins for the cluster Phill is using. We do download the standard cert bundles for the cluster. So I’m surprised that its not finding the cert. Is there a specific bundle or version of the bundles we should be pulling? We are using Rocky 8 and we have:

[root@holy8a24507 general]# rpm -qa | grep ca-cert
ca-certificates-2024.2.69_v8.0.303-80.0.el8_10.noarch

As I’m happy to add the appropriate cert if I know which RPM to pull.

6 Likes
14

@Paul_Edmon We are using rustls-native-certs on Linux for Juliaup to handle https certificate validation. Is there any info in that README that might help?

And just to rule out this option: you don’t have any TLS terminating proxy or anything like that between the cluster and the internet, right?

4 Likes
15

We don’t. It should be wide open.

What I do see in that doc you sent is:

On all platforms, the SSL_CERT_FILE environment variable is checked first. If that’s set, certificates are loaded from the path specified by that variable, or an error is returned if certificates cannot be loaded from the given path. If it’s not set, then the platform-specific certificate source is used.

We aren’t setting that but we are setting SSL_CERT_DIR. It shouldn’t be too hard to set SSL_CERT_FILE as well. I will look into that.

2 Likes
16

Yup, that fixed it. Thanks for the assist.

12 Likes