# Using Julia in a restricted environment with Docker

**URL:** <https://discourse.julialang.org/t/using-julia-in-a-restricted-environment-with-docker/12179>\
**Category:** General Usage\
**Created:** [July 5, 2018, 1:30pm UTC](https://discourse.julialang.org/t/using-julia-in-a-restricted-environment-with-docker/12179 "2018-07-05T13:30:52Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![mlhetland](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mlhetland/32/283_2.png) [@mlhetland](https://discourse.julialang.org/u/mlhetland)\
**Post date:** [July 5, 2018, 1:30pm UTC](https://discourse.julialang.org/t/using-julia-in-a-restricted-environment-with-docker/12179/1 "2018-07-05T13:30:53Z")

</div>

In my algorithms course, we plan to transition to using Julia this fall, and we are currently trying to make it work with [INGInious](https://github.com/UCL-INGI/INGInious), a system that automatically runs and tests code or code fragments submitted by students, using Docker containers. The main issue we keep facing is giving Julia access to the files it wants – primarily the `~/.julia` directory. We’ve managed to solve this by setting `HOME` to a directory where the process has read and write access, which solves some problems. For example, we are now allowed to use the ordinary `using` mechanism (not possible before we adjusted `HOME`), as the relevant calls to `stat` seem to be permitted. And I guess this would also permit compilation caching of any modules the students wrote themselves and submitted. However, we’d also want to install external modules for the students to use – so I guess a pre-populated `.julia` directory would have to be copied into the relevant directory/container for each run?

We could place it in a shared directory with no write access, but that would prevent the students from precompiling their own modules. Possibly not a real issue (we’re mostly talking small code fragments here anyway), but it doesn’t seem ideal. It would be great if we could have multiple locations – one with precompiled globally installed/shared modules and one with the compiler cache for the local code of the students themselves. I haven’t seen any such functionality in `Pkg3`, but maybe I missed something? (I guess the reason this works for, e.g., Python is that it caches the compiled files alongside the source.)

Does anyone else have any experience with running Julia in a related context? Are there any other issue to consider (and any relevant solutions)?

---

<div class="post-metadata">

**Author:** ![mlhetland](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mlhetland/32/283_2.png) [@mlhetland](https://discourse.julialang.org/u/mlhetland)\
**Post date:** [July 5, 2018, 1:37pm UTC](https://discourse.julialang.org/t/using-julia-in-a-restricted-environment-with-docker/12179/2 "2018-07-05T13:37:04Z")

</div>

Hm. I now see that there _is_ something we might be able to use, i.e., `JULIA_DEPOT_PATH`. I guess I should have read the docs more carefully! (Not 100% sure this solves everything, but still…)

---

<div class="post-metadata">

**Author:** ![gbenatt92](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gbenatt92/32/4583_2.png) [@gbenatt92](https://discourse.julialang.org/u/gbenatt92)\
**Post date:** [July 5, 2018, 1:46pm UTC](https://discourse.julialang.org/t/using-julia-in-a-restricted-environment-with-docker/12179/3 "2018-07-05T13:46:04Z")

</div>

I think you can find some ideas on how to set it with Docker [here](https://github.com/jupyter/docker-stacks/blob/master/datascience-notebook/Dockerfile#L22).

I’m using this image myself to learn Julia and it’s proving really helpful.

---

<div class="post-metadata">

**Author:** ![quinnj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/quinnj/32/11_2.png) [@quinnj](https://discourse.julialang.org/u/quinnj)\
**Post date:** [July 5, 2018, 1:51pm UTC](https://discourse.julialang.org/t/using-julia-in-a-restricted-environment-with-docker/12179/4 "2018-07-05T13:51:15Z")

</div>

[https://gist.github.com/quinnj/169a2d712ab8f780e56772ed88867d3a](https://gist.github.com/quinnj/169a2d712ab8f780e56772ed88867d3a)

Here’s the “base” Dockerfile I use w/ a few different custom projects. Basically it sets up a “default” user and uses shared `Project.toml`, `Manifest.toml` files to install global/shared external packages.

Custom applications can then just use this docker image as the base image, copy their own `Project.toml`/`Manifest.toml` in, instantiate their project and be on their way. julia is on the path (installed in `/usr/local/bin`), so that’s convenient.

Happy to answer any other questions about this setup.

---

<div class="post-metadata">

**Author:** ![ExpandingMan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/expandingman/32/866_2.png) [@ExpandingMan](https://discourse.julialang.org/u/ExpandingMan)\
**Post date:** [July 5, 2018, 1:55pm UTC](https://discourse.julialang.org/t/using-julia-in-a-restricted-environment-with-docker/12179/5 "2018-07-05T13:55:43Z")

</div>

It would be really nice if this were available somewhere that makes it easy to find. This even seems like something that should appear in the official Julia documentation somewhere as containers becoming increasingly more important these days.

---

<div class="post-metadata">

**Author:** ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)\
**Post date:** [July 5, 2018, 3:55pm UTC](https://discourse.julialang.org/t/using-julia-in-a-restricted-environment-with-docker/12179/6 "2018-07-05T15:55:52Z")

</div>

I’ve experimented with this as well, see [https://github.com/JuliaGPU/docker](https://github.com/JuliaGPU/docker), where I specify a volume for installed packages and one for the current working directory (as well as a more complicated template scheme to provide preinstalled packages and build them upon first run). Might be a bit over-engineered, and I don’t plan to maintain it now that GPU packages are easier to install.

---

<div class="post-metadata">

**Author:** ![marius311](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marius311/32/3953_2.png) [@marius311](https://discourse.julialang.org/u/marius311)\
**Post date:** [July 5, 2018, 4:25pm UTC](https://discourse.julialang.org/t/using-julia-in-a-restricted-environment-with-docker/12179/7 "2018-07-05T16:25:33Z")

</div>

I’ve used Julia in Docker for reproducibility and for running on MyBinder a few times. I find it works fairly well. You can pre-install and precompile packages in the Docker container by including something like this in your Dockerfile (assuming Julia 0.6):

```nohighlight
COPY REQUIRE $HOME/REQUIRE
RUN julia -e "Pkg.init(); mv(\"$HOME/REQUIRE\",Pkg.dir(\"REQUIRE\"),remove_destination=true); Pkg.resolve(); for p in Pkg.available(); try @eval using \$(Symbol(p)); println(p); end; end"

```

where you have listed all your dependent packages in the `REQUIRE` file.

---

<div class="post-metadata">

**Author:** ![mlhetland](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mlhetland/32/283_2.png) [@mlhetland](https://discourse.julialang.org/u/mlhetland)\
**Post date:** [July 6, 2018, 11:10am UTC](https://discourse.julialang.org/t/using-julia-in-a-restricted-environment-with-docker/12179/8 "2018-07-06T11:10:22Z")

</div>

Thanks for lots of great answers! Something along these lines would indeed (as @ExpandingMan suggests) be a great tutorial or documentation section. But if nothing else, I guess people can at least find this thread, now 😃

---

<div class="post-metadata">

**Author:** ![johnh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johnh/32/3615_2.png) [@johnh](https://discourse.julialang.org/u/johnh)\
**Post date:** [July 6, 2018, 12:27pm UTC](https://discourse.julialang.org/t/using-julia-in-a-restricted-environment-with-docker/12179/9 "2018-07-06T12:27:22Z")

</div>

> [@mlhetland](#):
>
> In my algorithms course, we plan to transition to using Julia this fall,

Yaaay 🎆🎉

---

<div class="post-metadata">

**Author:** ![mlhetland](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mlhetland/32/283_2.png) [@mlhetland](https://discourse.julialang.org/u/mlhetland)\
**Post date:** [August 16, 2018, 8:50pm UTC](https://discourse.julialang.org/t/using-julia-in-a-restricted-environment-with-docker/12179/10 "2018-08-16T20:50:37Z")

</div>

Hm. We had things working with a 0.7 nightly, but now (with 1.0, and with 0.7 final, it seems), we again get access errors – presumably because precompilation is trying to access something it shouldn’t. Is there any easy way to avoid that? (It seems `Pkg` mainly uses `DEPOT_PATH[1]`, but I also see activity in temp-directories, and – when installing things, which may not be that relevant, it also reads `~/.gitconfig`, for example, presumably via `libgit2`.)