# Installing packages via an SSH SOCKS proxy on a compute cluster

**URL:** <https://discourse.julialang.org/t/installing-packages-via-an-ssh-socks-proxy-on-a-compute-cluster/71735>\
**Category:** General Usage\
**Tags:** proxy\
**Created:** [November 18, 2021, 5:34pm UTC](https://discourse.julialang.org/t/installing-packages-via-an-ssh-socks-proxy-on-a-compute-cluster/71735 "2021-11-18T17:34:02Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![sloede](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sloede/32/44787_2.png) [@sloede](https://discourse.julialang.org/u/sloede)\
**Post date:** [November 18, 2021, 5:34pm UTC](https://discourse.julialang.org/t/installing-packages-via-an-ssh-socks-proxy-on-a-compute-cluster/71735/1 "2021-11-18T17:34:02Z")

</div>

I have spent the better part of a day figuring this out, so I thought it might be interesting to share for others:

In my case, I would like to run Julia on a university cluster with very restricted internet access from inside. That is, most websites and services are not directly reachable from the login nodes (I am not even talking about the compute nodes). This makes it impossible to, e.g., just run Julia and install packages using `Pkg`, since the package servers and other locations are not available.

If you are using a Linux client, witha _fairly_ recent OpenSSH version (\>= 7.6, i.e., post-2017), and have SSH access to the remote machine, here is one possible solution:

First, connect to the cluster via SSH and enable reverse dynamic forwarding with

```bash
ssh -R 12345 mycluster

```

This will create a SOCKS4/5 proxy on the _remote_ machine `mycluster` and have it listen on port `12345`. All connections _originating on the the remote machine_ that connect to this proxy will then be forwarded to the internet via your _local_ machine.

To make Julia use the proxy, you need to set the environment variables `HTTP_PROXY` and `HTTPS_PROXY` to use the SOCKS proxy, e.g., by starting Julia as

```bash
HTTP_PROXY=socks5://localhost:12345 HTTPS_PROXY=socks5://localhost:12345 julia

```

Now, all normal package operations should work as usual.

In case you want to make this setup permanent, you can add an entry to your local clients `~/.ssh/config` file:

```nohighlight
Host mycluster
  RemoteForward 12345

```

This will save you from typing the `-R 12345` each time.

You can instruct Julia to always use the SOCKS proxy by adding the following lines to your `~/.julia/config/startup.jl` file on the _remote_ machine, saving you from setting the environment variables at each start:

```julia
ENV["HTTP_PROXY"] = "socks5://localhost:12345"
ENV["HTTPS_PROXY"] = "socks5://localhost:12345"

```

If someone other than you already uses the hardcoded port, you need to override it again by providing the respective arguments on the command line.

Thanks to [Install packages behind the proxy](https://discourse.julialang.org/t/install-packages-behind-the-proxy/23298) for their discussions on how to set and use proxy servers with Julia.  
Thanks to @vchuravy @giordano @simonbyrne for their helpful suggestions and discussions.

---

<div class="post-metadata">

**Author:** ![MirekKratochvil](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mirekkratochvil/32/21851_2.png) [@MirekKratochvil](https://discourse.julialang.org/u/MirekKratochvil)\
**Post date:** [May 3, 2022, 7:55am UTC](https://discourse.julialang.org/t/installing-packages-via-an-ssh-socks-proxy-on-a-compute-cluster/71735/2 "2022-05-03T07:55:11Z")

</div>

Hi!  
I tried doing the same on 1.8.0-beta3 and I’m getting the following error:

> ERROR: failed to clone from [GitHub - JuliaRegistries/General: The official registry of general Julia packages](https://github.com/JuliaRegistries/General.git), error: GitError(Code:ERROR, Class:HTTP, unknown http scheme ‘socks5’)

I assume that the git version does not support socks5, although the system git actually can do socks5 (tested manually).

Is there any way to work that around?

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [May 3, 2022, 8:04am UTC](https://discourse.julialang.org/t/installing-packages-via-an-ssh-socks-proxy-on-a-compute-cluster/71735/3 "2022-05-03T08:04:11Z")

</div>

> [@MirekKratochvil](#):
>
> although the system git actually can do socks5 (tested manually).

Then you can try setting the environment variable [`JULIA_PKG_USE_CLI_GIT=true`](https://docs.julialang.org/en/v1.9-dev/manual/environment-variables/#JULIA_PKG_USE_CLI_GIT)

---

<div class="post-metadata">

**Author:** ![MirekKratochvil](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mirekkratochvil/32/21851_2.png) [@MirekKratochvil](https://discourse.julialang.org/u/MirekKratochvil)\
**Post date:** [May 3, 2022, 8:11am UTC](https://discourse.julialang.org/t/installing-packages-via-an-ssh-socks-proxy-on-a-compute-cluster/71735/4 "2022-05-03T08:11:04Z")

</div>

@giordano thanks a lot, using the system git works perfectly!

---

<div class="post-metadata">

**Author:** ![b-fg](https://avatars.discourse-cdn.com/v4/letter/b/848f3c/32.png) [@b-fg](https://discourse.julialang.org/u/b-fg)\
**Post date:** [September 27, 2022, 3:31pm UTC](https://discourse.julialang.org/t/installing-packages-via-an-ssh-socks-proxy-on-a-compute-cluster/71735/5 "2022-09-27T15:31:42Z")

</div>

This worked for me too, thanks for sharing!

---

<div class="post-metadata">

**Author:** ![sloede](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sloede/32/44787_2.png) [@sloede](https://discourse.julialang.org/u/sloede)\
**Post date:** [September 27, 2022, 3:46pm UTC](https://discourse.julialang.org/t/installing-packages-via-an-ssh-socks-proxy-on-a-compute-cluster/71735/6 "2022-09-27T15:46:17Z")

</div>

You’re welcome! In the meantime, I found an even better approach for specifying the HTTP/HTTPS proxies: Instead of using the environment variables `HTTP_PROXY` and `HTTPS_PROXY`, I recommend to set the _lowercase_ variants `http_proxy` and `https_proxy`, e.g., in your `~/.bashrc` file:

```bash
export http_proxy=socks5://localhost:12345
export https_proxy=socks5://localhost:12345

```

This works with Julia and `Pkg`, but as an added bonus it also works with `curl`! For example, to download the latest stable Julia version, you can now just execute

```bash
curl -OJL https://julialang-s3.julialang.org/bin/linux/x64/1.8/julia-1.8.1-linux-x86_64.tar.gz

```

Unfortunately I cannot edit the original post anymore, but I thought I’d share this here anyways.

---

<div class="post-metadata">

**Author:** ![zhubonan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/zhubonan/32/31877_2.png) [@zhubonan](https://discourse.julialang.org/u/zhubonan)\
**Post date:** [November 18, 2022, 9:21am UTC](https://discourse.julialang.org/t/installing-packages-via-an-ssh-socks-proxy-on-a-compute-cluster/71735/7 "2022-11-18T09:21:35Z")

</div>

Note that in some machines without even DNS access, one will need

```julia
export http_proxy=socks5h://localhost:12345
export https_proxy=socks5h://localhost:12345

```

which allows resolving domain names via socks5.