# Deployment at scale
**URL:** https://discourse.julialang.org/t/deployment-at-scale/38536
**Category:** Tooling
**Created:** [May 1, 2020, 9:11am UTC](https://discourse.julialang.org/t/deployment-at-scale/38536 "2020-05-01T09:11:48Z")
**Posts on this page:** 9
**Page:** 1
### Author:  [@Emmanuel-R8](https://discourse.julialang.org/u/Emmanuel-R8)
#### Post date: [May 1, 2020, 9:11am UTC](https://discourse.julialang.org/t/deployment-at-scale/38536/1 "2020-05-01T09:11:48Z")
Earlier this week I made a presentation of Julia for data science and ML. One question came up to which I could only give a non-committal answer. Paraphrasing, once our notebooks work fine, how do we deploy at scale on various infrastructure solution.
I did a bit of research and came up with:
- Out of the box, without additional packages, julia can be executed on several worker processes on a single machine or several machines connected through ssh.
- [GitHub - staticfloat/julia-docker: Various Dockerfiles for Julia](https://github.com/staticfloat/julia-docker) has a number of of docker files.
- There is a community focused of running Julia on parallel workloads ([JuliaParallel · GitHub](https://github.com/JuliaParallel)). In particular, they oversee MPI wrappers and another package to deploy on many standard batch systems (e.g. Slurm).
- Dokku ([Dokku - The smallest PaaS implementation you've ever seen](http://dokku.viewdocs.io/dokku/)) and Elasticluster ([https://elasticluster.readthedocs.io/en/](https://elasticluster.readthedocs.io/en/)) have examples for Julia
- The most prominent Julia consultancy company (Julia Computing) has a proprietary solution call JuliaRun.
However, I have not found a central repository of knowledge, success story or clear howtos / workflows for handling that.
Are you aware of anything? Can anybody comment on how mature those options are for a production, high quality environment (think paranoid financial trading requirements)?
Thanks.
---
### Author:  [@johnh](https://discourse.julialang.org/u/johnh)
#### Post date: [May 1, 2020, 12:03pm UTC](https://discourse.julialang.org/t/deployment-at-scale/38536/2 "2020-05-01T12:03:11Z")
This is a very good discussion.
The ‘classic’ HPC answer is to us ea job scheduler such as Slurm.
I guess I would ask them what their current method of deploying Docker containers is.
The answer is likely to be Kubernetes.
Personally I rather like Singularity containers - which are inherently secure and you can ‘read in’ a Docker container
> **[SingularityCE Documentation Hub](https://sylabs.io/docs/)**
>
> Access guides for all versions of SingularityCE. Get everything you need to leverage the free and open-source container platform.
I rather like the concepts of Nomad also, and Singularity fits with it
> **[Nomad by HashiCorp](https://www.nomadproject.io/)**
>
> Nomad is a simple and flexible scheduler and orchestrator for managing containers and non-containerized applications across on-prem and clouds at scale.
---
### Author:  [@johnh](https://discourse.julialang.org/u/johnh)
#### Post date: [May 1, 2020, 12:08pm UTC](https://discourse.julialang.org/t/deployment-at-scale/38536/3 "2020-05-01T12:08:15Z")
If you are working in the deep learning field there are several frameworks for deploying models - I guess most assume Kubernetes.
For example Seldon [Tech Ethics Meetup: AI, Data and Ethics with Prof. Joanna Bryson - Seldon](https://www.seldon.io/tech/)
Hopsworks [https://www.logicalclocks.com/](https://www.logicalclocks.com/)
I am unaware of Seldon/Hopsworks being used with Julia
---
### Author:  [@johnh](https://discourse.julialang.org/u/johnh)
#### Post date: [May 1, 2020, 12:09pm UTC](https://discourse.julialang.org/t/deployment-at-scale/38536/4 "2020-05-01T12:09:42Z")
Pushign Singularity again, it has signed containers, which may be important in security onscious environments
[https://sylabs.io/guides/3.5/user-guide/signNverify.html](https://sylabs.io/guides/3.5/user-guide/signNverify.html)
---
### Author:  [@ffevotte](https://discourse.julialang.org/u/ffevotte)
#### Post date: [May 1, 2020, 6:57pm UTC](https://discourse.julialang.org/t/deployment-at-scale/38536/5 "2020-05-01T18:57:52Z")
Julia Computing recently proposed a webinar on [“Building Production Applications Using Julia”](https://www.linkedin.com/posts/juliacomputing_register-activity-6658395412136849408-sNaN/), led by @avik and which covered some aspects related to deployment. Hopefully a recorded version will soon be available [online](https://juliacomputing.com/resources/webinars).
---
### Author:  [@ffevotte](https://discourse.julialang.org/u/ffevotte)
#### Post date: [May 1, 2020, 7:15pm UTC](https://discourse.julialang.org/t/deployment-at-scale/38536/6 "2020-05-01T19:15:41Z")
> [@Emmanuel-R8](#):
>
> Can anybody comment on how mature those options are for a production, high quality environment
I’ve recently had the occasion/opportunity to help a company put a Julia code into production, and we found the tooling we used to be very mature. Nothing really fancy, but things that work:
1. a fresh docker image is built
2. the Julia application is installed in the docker image (simply using `Pkg`)
3. …and compiled there using `PackageCompiler`
4. tests are run inside the docker image to check that everything works (`Pkg` again)
5. all this workflow is triggered in the CI/CD system
6. the docker images can then be deployed either on local resources or on cloud-computing platforms
As you can see, the entire workflow was built upon `Pkg` and `PackageCompiler` (v1) which we found to work very reliably.
---
### Author:  [@Emmanuel-R8](https://discourse.julialang.org/u/Emmanuel-R8)
#### Post date: [May 2, 2020, 6:32am UTC](https://discourse.julialang.org/t/deployment-at-scale/38536/7 "2020-05-02T06:32:20Z")
Thanks.
In other posts, many have pointed out that the initial compile time on spinning out a docker image was an issue. Does PackageCompiler address that completely? What about `snapshotted ` VM image ready to go?
---
### Author:  [@ffevotte](https://discourse.julialang.org/u/ffevotte)
#### Post date: [May 2, 2020, 2:45pm UTC](https://discourse.julialang.org/t/deployment-at-scale/38536/8 "2020-05-02T14:45:25Z")
> [@Emmanuel-R8](#):
>
> In other posts, many have pointed out that the initial compile time on spinning out a docker image was an issue. Does PackageCompiler address that completely?
In my experience, with a good `precompile_execution_file` (not always easy to provide), the time needed to spawn a new Julia process and load all packages in the environment is reduced to something like 1s (max). The first run of every function might sometimes still be a bit slower than usual if additional compilation is required, for functions which could not be captured in the system image. But at least that mostly eliminates the latency problem.
I guess everything depends on the use case, and especially the expected run time of the Julia process.
> [@Emmanuel-R8](#):
>
> What about `snapshotted ` VM image ready to go?
That might very well be a good idea, but I never tried…
---
### Author:  [@onetonfoot](https://discourse.julialang.org/u/onetonfoot)
#### Post date: [May 2, 2020, 4:02pm UTC](https://discourse.julialang.org/t/deployment-at-scale/38536/9 "2020-05-02T16:02:56Z")
If you choose the docker route then [SimpleContainerGenerator.jl](https://github.com/bcbi/SimpleContainerGenerator.jl) could be useful. I’m yet to try it out but looks like it will automate a lot of the boilerplate