Creating a Docker Base Image for Faster Deployments

mrufsvold
Thu, 10 Oct 2024 20:35:02 +0000
Tooling

In my experience, deploying Julia to the cloud can be frustrating because it can be slow to build the container used to your app. I don’t seem to be the only one. A major gripe is that AWS CodeBuild can’t do layer caching in a VPC, so strategies for rearranging your DockerFile to benefit from cache don’t help.

I finally got a working solution for me. I build a base image with a SysImage of my dependencies and then copy my current src directory on deployment. Posting it here in case it is helpful for others. It comes in four parts
1. A Julia script for building a system image within the Docker container (click for more details) 2. A DockerFile to build the base image (click for more details) 3. A bash script to orchestrate building the base image when you change your dependencies (click for more details) 4. The DockerFile to use in deployment (click for more details)

Note that all these files assume they are at the root of your project directory.

Thanks to @oxinabox and others for your help along the way!

UPDATE: I discovered that, even if you copy in a new src/ directory, if you import your project package with using MyPackage it will pick up the version that PackageCompiler.jl cached in the sys image. So it’s best to have your entry_point_script.jl use include("src/MyPackage.jl"), so that you are using the fresh version of your package.

2 posts - 2 participants