Install packages on machine not connected to internet

General Usage
,
1

Hi,

I work on a windows machine connected to internet. (I can’t use Docker on it)
From that machine I can connect to a ubuntu server which is not connected to internet.
I manage to install Julia on the server by downloading the right binary on the windows machine and moving it (and unzipping it) on the ubuntu server.

The issue is that I would like to install packages on the ubuntu server. Does anyone know how I can do that?

Is there a way to download a package (example: DataFrames.jl) and all its dependencies (including the right jll packages) and then move them to the server ?

Thank you

1 Like
2

I work on secure (dark) sites also.
One thing to ask - I had a customer who assured me that they had a secure network. It turns out hey have a proxy so can easily install software.
So ask if there is a proxy.

3

Lol, it is just my IT which is a pain in the back. It is a dev server which is not allowed to be connected to internet or any production environment. On windows I do indeed need a proxy to access internet but on the server I am not aware of anything and if I ask they will likely chop my head off.

1 Like
4

I have no idea if this could help: LocalPackageServer

5

Sort of, but it has no real advantages over GitHub - JuliaPackaging/PkgServer.jl for this purpose. See JuliaRegistries setup for air gapped network - #21 by GunnarFarneback for some earlier discussion in this area. I should add that to get the right artifacts you should populate from the same platform you use on the air-gapped network.

6

You can try the following.

First, on the machine which has internet connection, run the following:

$ JULIA_DEPOT_PATH=offline_depot julia
julia> ]
pkg> add StatsBase MultivariateStats # these are just examples, add all the packages you will need here
pkg> <Ctrl-D>

Then copy the offline_depot directory (created by Julia in the previous step) onto the machine you will use in offline mode. Then start Julia like this:

$ JULIA_DEPOT_PATH=<path-to-your>/offline_depot JULIA_PKG_OFFLINE=true julia
julia> using MultivariateStats # use the preinstalled packages here

Hope that helps.

3 Likes
7

Thanks but that only works if both machines are running the same operating system.

8

Right, sorry, I ignored that part of the question.

I assume you connect to the Ubuntu server via ssh. If that is the case, you can set up ssh as a proxy server and use it to access the internet from the server.

windows> ssh -R8080 ubuntu-server.local # edited from: ssh -D8080 ubuntu-server.local
ubuntu$ git config --global http.proxy 'socks5://127.0.0.1:8080'
ubuntu$ export JULIA_PKG_USE_CLI_GIT=true
ubuntu$ julia
julia> ]registry up

I haven’t tested this, bit it should work, as long as you fetch registries and packages using HTTP.

Alternatively, you can set up WSL on your Windows machine, install Julia there, and then you are on the same platform as your server, so the depot copying solution will work.

3 Likes
9

Maybe it works with an ssh tunnel and port forwarding, but I’ve never tried this myself.

EDIT: This is essentially the same approach as the one described by @HanD above.

1 Like
10

I get the following error for both @HanD and @jbrea suggestions:

(@v1.9) pkg> registry up
┌ Warning: could not download https://pkg.julialang.org/registries
│   exception = RequestError: Could not resolve host: pkg.julialang.org while requesting https://pkg.julialang.org/registries
└ @ Pkg.Registry ~/julia/share/julia/stdlib/v1.9/Pkg/src/Registry/Registry.jl:69

(@v1.9) pkg> add DataFrames
  Installing known registries into `~/.julia`
┌ Warning: could not download https://pkg.julialang.org/registries
│   exception = RequestError: Could not resolve host: pkg.julialang.org while requesting https://pkg.julialang.org/registries
└ @ Pkg.Registry ~/julia/share/julia/stdlib/v1.9/Pkg/src/Registry/Registry.jl:69
┌ Warning: could not download https://pkg.julialang.org/registries
│   exception = RequestError: Could not resolve host: pkg.julialang.org while requesting https://pkg.julialang.org/registries
└ @ Pkg.Registry ~/julia/share/julia/stdlib/v1.9/Pkg/src/Registry/Registry.jl:69
     Cloning registry from "https://github.com/JuliaRegistries/General.git"
fatal: unable to access 'https://github.com/JuliaRegistries/General.git/': Failed to connect to 127.0.0.1 port 8080: Connection refused
11

My bad, -D proxies local connections to the remote server, you need the opposite, proxy remote connections to the local machine with -R. So the correct ssh command would look like this:

$ ssh -R8080 ubuntu-server.local

I now tested this in my own local environment, and it seems to do the trick.

1 Like
12

When I run this on the windows machine I get:

Warning: remote port forwarding failed for listen port 8080

Then it asks me for my password, which I enter and then connect. When I try to install a package I still get the same error Connection refused

13

I think the issue is port forwardimg is not allowed. In the following file on the ubuntu machine: /etc/ssh/sshd_config I found that AllowTcpForwarding no and I am not going to change that file otherwise I will really get my head chopped off… Is that something you can test on your side if you set that variable? Thank you

14

Doesn’t solve the Connection Refused problem, but you can eliminate the password login by setting up PKI for SSH:

1 Like
15

Maybe this helps?

16

It’s similar to what had been suggested above, but this thread

has been around for about a couple of years.

17

The 8080 port is probably taken by some other process. You can use any number between 1024 and 65535. Just make sure to use the same number in the git config.

18

But why git? Registered packages aren’t installed via git.

19

I have checked with netstat and the port is not used. I have tried with various other port and I get the same error.

20

Thanks but I am a bit struggling with the exact steps.