Cannot use JuliaHub package server

General Usage
1

I am behind a large-corp firewall, and following this advice I am trying JuliaHub. I made an account, downloaded the auth token:

PS C:\Users\Cedric> dir .julia/servers/juliahub.com/auth.toml                                                                                                                                                                                                                                                                                                             

C:\Users\Cedric\.julia\servers\juliahub.com                                                                                                                                                                                                                                                                                                         
Mode                 LastWriteTime         Length Name                                                                  
----                 -------------         ------ ----                                                                  
-a----        2023/06/02      9:58           2097 auth.toml

but somehow this isn’t enough

julia> ENV["JULIA_PKG_SERVER"] = "https://pkg.juliahub.com"                                                             
"https://pkg.juliahub.com"                                                                                                                                                                                                                                    
(@v1.9) pkg> add ScikitLearnBase                                                                                        
┌ Warning: could not download 
https://pkg.juliahub.com/registries                                                       
│   exception = RequestError: HTTP/1.1 401 
Unauthorized while requesting 
https://pkg.juliahub.com/registries            â”” @ 
Pkg.Registry 
C:\Users\Cedric\AppData\Local\Programs\Julia-
1.9.0\share\julia\stdlib\v1.9\Pkg\src\Registry\Registry.jl:69                                                                                                                       
Updating registry at 
`C:\Users\Cedric\.julia\registries\General.toml`                                                
Resolving package versions...                                                                                        
┌ Warning: could not download https://pkg.juliahub.com/registries                                                       
│   exception = RequestError: HTTP/1.1 401 
Unauthorized while requesting https://pkg.juliahub.com/registries            â””

Am I doing anything wrong?

2

My first guess is that you might have to change .julia/servers/juliahub.com/auth.toml to .julia/servers/pkg.juliahub.com/auth.toml.

But, in general, instead of managing the auth.toml files manually, I prefer to just use PkgAuthentication.jl:

4 Likes
3

That worked! Thank you. Unfortunately, I get the same hash mismatch problem installing artifacts as I did before on Julia’s default servers…

(@v1.9) pkg> add GR                                                                                                        
Resolving package versions...                                                                                         
Downloading artifact: GR                                                                                                 
Downloaded artifact: GR                                                                                                
Downloading artifact: GR                                                                                               
┌ Error: Hash Mismatch!                                                                                                 
│   Expected sha256:   8cbef895c666e99d6d9e1790cdd0f2eb44f950a43a1ad76f179a16166d53ff70                                 
│   Calculated sha256: 06c8a78b874f739e825ccdeef2a0cc478dcd8b7b9144e45ba188dd4cb3745fcf                                 
â”” @ Pkg.PlatformEngines C:\Users\Cedric\AppData\Local\Programs\Julia-
1.9.0\share\julia\stdlib\v1.9\Pkg\src\PlatformEng  Downloaded artifact: GR                                                                                               
ERROR: Unable to automatically download/install artifact 'GR' from sources listed in 'C:\Users\Cedric\.julia\packages\GR_jll\0C12V\Artifacts.toml'.                                                                                           
Sources attempted:                                                                                                      
- https://pkg.juliahub.com/artifact/9dbefe9cacd3199aed3c251e7ecbccc93ca86d2f                                                
Error: This does not appear to be a TAR file/stream — invalid version string for tar file: "aP". Note: Tar.jl does not handle decompression; if the tarball is compressed you must use an external command like `gzcat` or package like CodecZlib.jl to decompress it. See the README file for examples.                                                            
- https://github.com/JuliaBinaryWrappers/GR_jll.jl/releases/download/GR-v0.72.7+0/GR.v0.72.7.x86_64-w64-mingw32-cxx11.tar.gz                                                                                                                        
Error: Verification failed. Download does not match expected hash

Somehow the firewall gets in the way, and delivers some kind of “This page is forbidden” HTML page instead of delivering the actual file, hence the hash mismatch.

4

I’ve tried downloading this same file from both juliahub.com and pkg.juliahub.com (both should work) using Downloads.download with the appropriate auth headers manually added and it works for me:

julia> using Downloads, Tar

julia> token = "secret auth token from ~/.julia/servers/juliahub.com/auth.toml";

julia> tmp = Downloads.download("https://juliahub.com/artifact/9dbefe9cacd3199aed3c251e7ecbccc93ca86d2f", headers=["Authorization" => "Bearer $token", "Julia-Version" => "1.9.1"])
"/var/folders/fm/ld6yc24s56q94q3n2wf20v7m0000gn/T/jl_yh0tA9CHUL"

julia> Tar.tree_hash(`gzcat $tmp`)
"9dbefe9cacd3199aed3c251e7ecbccc93ca86d2f"

You could try the same with your auth token value. You might need to refresh it, which the Pkg logic takes care of automatically. That could at least allow debugging what content you’re getting that doesn’t look like a valid tarball.

I’m also not sure why the second download would be failing since I’ve downloaded that binary from GitHub and gotten the correct SHA256 hash. But it shouldn’t even be getting to that point since the first download from JuliaHub should work.

1 Like
5

Hi Stefan, thank you for looking into it. We found a way to temporarily bypass the firewall, and all is good for now. Julia has entered big-corp :slightly_smiling_face:

3 Likes