# Package install on (offline) Cluster - Build needs internet

**URL:** https://discourse.julialang.org/t/package-install-on-offline-cluster-build-needs-internet/19862
**Category:** Julia at Scale
**Tags:** question, cluster
**Created:** [January 21, 2019, 9:24am UTC](https://discourse.julialang.org/t/package-install-on-offline-cluster-build-needs-internet/19862 "2019-01-21T09:24:02Z")
**Posts on this page:** 8
**Page:** 1

<div class="post-metadata">

### Author: ![JonasIsensee](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jonasisensee/32/4704_2.png) [@JonasIsensee](https://discourse.julialang.org/u/JonasIsensee)
#### Post date: [January 21, 2019, 9:24am UTC](https://discourse.julialang.org/t/package-install-on-offline-cluster-build-needs-internet/19862/1 "2019-01-21T09:24:03Z")

</div>

Hey everyone,

I’m trying to use julia 1.0 on our cluster.  
It is not connected to the internet (login via login-server)  
so I can’t use the standard procedure to install the packages I need.

I read [here](https://discourse.julialang.org/t/offline-package-installation/7880) that one option should  
be able to set up the `.julia` folder on a separate computer and copy it to the cluster.  
(But maybe that was only true for pre-1.0?)

I tried it but got the following error

```julia
ERROR: LoadError: InitError: /home/isensee/.julia/packages/SpecialFunctions/fvheQ/deps/usr/lib/libopenspecfun.so cannot be opened, 
Please re-run Pkg.build("SpecialFunctions"), and restart Julia.

```

When I try to follow the order I get

```julia
(v1.0) pkg> build SpecialFunctions
  Updating registry at `~/.julia/registries/General`
  Updating git-repo `https://github.com/JuliaRegistries/General.git`
┌ Warning: Some registries failed to update:
│ — /home/isensee/.julia/registries/General — failed to fetch from repo
└ @ Pkg.API /usr/lfpn/SOURCES.ORIGINAL/julia/usr/share/julia/stdlib/v1.0/Pkg/src/API.jl:144
  Building SpecialFunctions → `~/.julia/packages/SpecialFunctions/fvheQ/deps/build.log`
┌ Error: Error building `SpecialFunctions`: 
│ [ Info: Downloading https://github.com/JuliaMath/OpenspecfunBuilder/releases/download/v0.5.3-3/Openspecfun.v0.5.3.x86_64-linux-gnu-gcc4.tar.gz to /home/isensee/.julia/packages/SpecialFunctions/fvheQ/deps/usr/downloads/Openspecfun.v0.5.3.x86_64-linux-gnu-gcc4.tar.gz...
│ ERROR: LoadError: Could not download https://github.com/JuliaMath/OpenspecfunBuilder/releases/download/v0.5.3-3/Openspecfun.v0.5.3.x86_64-linux-gnu-gcc4.tar.gz to /home/isensee/.julia/packages/SpecialFunctions/fvheQ/deps/usr/downloads/Openspecfun.v0.5.3.x86_64-linux-gnu
-gcc4.tar.gz:
│ ErrorException("")
│ Stacktrace:
│ [1] error(::String) at ./error.jl:33
│ [2] macro expansion at ./logging.jl:313 [inlined]
│ [3] #download#89(::Bool, ::Function, ::String, ::String) at /home/isensee/.julia/packages/BinaryProvider/4F5Hq/src/PlatformEngines.jl:487
│ [4] #download at ./none:0 [inlined]
│ [5] #download_verify#90(::Bool, ::Bool, ::Bool, ::Function, ::String, ::String, ::String) at /home/isensee/.julia/packages/BinaryProvider/4F5Hq/src/PlatformEngines.jl:567
│ [6] #download_verify at ./none:0 [inlined]
│ [7] #install#129(::Prefix, ::String, ::Bool, ::Bool, ::Bool, ::Function, ::String, ::String) at /home/isensee/.julia/packages/BinaryProvider/4F5Hq/src/Prefix.jl:314
│ [8] (::getfield(BinaryProvider, Symbol("#kw##install")))(::NamedTuple{(:prefix, :force, :verbose),Tuple{Prefix,Bool,Bool}}, ::typeof(install), ::String, ::String) at ./none:0
│ [9] top-level scope at /home/isensee/.julia/packages/SpecialFunctions/fvheQ/deps/build.jl:142
│ [10] include at ./boot.jl:317 [inlined]
│ [11] include_relative(::Module, ::String) at ./loading.jl:1038
│ [12] include(::Module, ::String) at ./sysimg.jl:29
│ [13] include(::String) at ./client.jl:388
│ [14] top-level scope at none:0
│ in expression starting at /home/isensee/.julia/packages/SpecialFunctions/fvheQ/deps/build.jl:139
│ [17:14:15] 
│ [17:14:15] curl: (7) Couldn't connect to server
└ @ Pkg.Operations /usr/lfpn/SOURCES.ORIGINAL/julia/usr/share/julia/stdlib/v1.0/Pkg/src/Operations.jl:1068

```

So it seems that a `Pkg.build` from a different computer is not compatible and rebuilding fails due  
to missing internet connection.  
Is there any way out?

---

<div class="post-metadata">

### Author: ![johnh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johnh/32/3615_2.png) [@johnh](https://discourse.julialang.org/u/johnh)
#### Post date: [January 21, 2019, 9:37am UTC](https://discourse.julialang.org/t/package-install-on-offline-cluster-build-needs-internet/19862/2 "2019-01-21T09:37:25Z")

</div>

@JonasIsensee I have been getting the ame error messages regardign the need to rebuild SpecialFunctions. I have been workign with the 1.1.0rc2 in a Singularity container environment.  
I think we need a discussion on installing Julia and installing packages on isolated systems.

---

<div class="post-metadata">

### Author: ![johnh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johnh/32/3615_2.png) [@johnh](https://discourse.julialang.org/u/johnh)
#### Post date: [January 21, 2019, 9:38am UTC](https://discourse.julialang.org/t/package-install-on-offline-cluster-build-needs-internet/19862/3 "2019-01-21T09:38:33Z")

</div>

I know this does not add to the discussion.  
But are you SURE your the login server on yoru cluster cannot download packages?  
If there is a proxy server you can set the HTTP\_PROXY environment variable and probably get it ti to work.

---

<div class="post-metadata">

### Author: ![JonasIsensee](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jonasisensee/32/4704_2.png) [@JonasIsensee](https://discourse.julialang.org/u/JonasIsensee)
#### Post date: [January 21, 2019, 9:56am UTC](https://discourse.julialang.org/t/package-install-on-offline-cluster-build-needs-internet/19862/4 "2019-01-21T09:56:24Z")

</div>

You are right.  
I can download the packages (`] add MyPackage`)  
from the login server.

Doing `] build SpecialFunctions ` on the login server  
does not solve my problem though as  
the cluster node still shows the same error as before.

I am trying to do `using MyPackage` on the login-server now  
( which we’re probably not supposed to do, as it’s a rather weak machine only meant for forwarding ssh sessions)  
but I’m not even sure if that’d make a difference

EDIT: It does not help. I still get

```julia
julia> using TimeseriesPrediction
[Info: Recompiling stale cache file /home/isensee/.julia/compiled/v1.0/TimeseriesPrediction/3Yac0.ji for TimeseriesPrediction [f218859d-9706-56aa-9ebf-1fa4ed7b8020]
ERROR: 
LoadError: InitError: /home/isensee/.julia/packages/SpecialFunctions/fvheQ/deps/usr/lib/libopenspecfun.so cannot be opened,
Please re-run Pkg.build("SpecialFunctions"), and restart Julia.

```

I don’t know of any proxy server but sent an email to see if one exists.

---

<div class="post-metadata">

### Author: ![JonasIsensee](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jonasisensee/32/4704_2.png) [@JonasIsensee](https://discourse.julialang.org/u/JonasIsensee)
#### Post date: [January 24, 2019, 5:03pm UTC](https://discourse.julialang.org/t/package-install-on-offline-cluster-build-needs-internet/19862/5 "2019-01-24T17:03:41Z")

</div>

It’s working now.

I talked to our admin and he didn’t know of a proper solution either  
so in the end he temporarily activated the internet access until  
all packages were installed.

As long as I don’t have to update anything I’ll be fine for the current project  
but this is really not a satisfactory solution.

---

<div class="post-metadata">

### Author: ![mopg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mopg/32/3368_2.png) [@mopg](https://discourse.julialang.org/u/mopg)
#### Post date: [July 20, 2019, 4:19pm UTC](https://discourse.julialang.org/t/package-install-on-offline-cluster-build-needs-internet/19862/6 "2019-07-20T16:19:32Z")

</div>

Did you ever find a solution for this? I’m running into the exact same issue and getting internet access is not an option for me.

---

<div class="post-metadata">

### Author: ![JonasIsensee](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jonasisensee/32/4704_2.png) [@JonasIsensee](https://discourse.julialang.org/u/JonasIsensee)
#### Post date: [July 20, 2019, 5:11pm UTC](https://discourse.julialang.org/t/package-install-on-offline-cluster-build-needs-internet/19862/7 "2019-07-20T17:11:13Z")

</div>

Hi @mopg,  
sadly no.  
The IT guy has probably forgotten that he wanted to turn off the internet access again.  
So I’m still installing packages the normal way.

---

<div class="post-metadata">

### Author: ![alirezamecheng](https://avatars.discourse-cdn.com/v4/letter/a/e95f7d/32.png) [@alirezamecheng](https://discourse.julialang.org/u/alirezamecheng)
#### Post date: [December 28, 2021, 7:25am UTC](https://discourse.julialang.org/t/package-install-on-offline-cluster-build-needs-internet/19862/8 "2021-12-28T07:25:10Z")

</div>

It seems too late for a response but since this question comes up in the searches I post here.  
I created a Julia program that mirrors the Julia repository for offline clusters. It is not an advanced code but it solved my problems. [nativeJuliaMirror.jl](https://github.com/alirezaghavaminia/nativeJuliaMirror.jl)
