Caching precompiled function

Usage First steps
#1

Hi,

I’m trying to figure out most convenient workflow/project structure for my use-case.

Currently I have a directory with multiple *.jl files without defining any new Modules or Packages.
This project is under continuous development and is being used on multiple devices.

On new device:

  1. install Julia
  2. git clone project
  3. julia packages.jl (acts like “py -m pip install -r requirements.txt” )

Now each time to run:

  1. git pull (if needed)
  2. julia
  3. include(“main.jl”)
  4. [1, 2, 3] |> main (user needs to specify arguments)

Only problem I have with this workflow is precompilation at include(“main.jl”):

@time include(“main.jl”)
19.197692 seconds (14.11 M allocations: 812.678 MiB, 1.66% gc time)

I don’t mind having dependencies in global environment. Don’t really need to define Module for namespace. This project will never be made public.

What would be the simplest way to reorganize my project to be able to cache precompiled main function.

#2

If you constantly update the file and want to get the new versions in the REPL, you might try includet from Revise.jl.

It als works with git updates like git pull.

But as you call it “Project” and “under development” you might also consider to create a module. Julia is able to instantiate packages very quickly. You could develop it locally and using FooPackacke and Revise can also track this package and updates.

#3

I have compiled some tips that appeared in these forums in previous posts here:

https://m3g.github.io/JuliaCookBook.jl/stable/workflow/

and here:

https://m3g.github.io/JuliaCookBook.jl/stable/modules/

But, as mentioned above, essentially the way to go is to use Revise.

2 Likes
#4

I do use Revise when changing code. I don’t want to precompile my code each time I start new session if there are no changes.

#5

I know you said you don’t need to define a module, but

Answer: turn it into a package.

6 Likes
#6

if you put the code in a package and you dev . it, then precompilation should only happen if the code changes. Otherwise it uses the cached version.

I’m not sure how that is realized in includet but my guess would be, that there is no precompilation happening

2 Likes
#7

By creating package I halved start up time. Load time now pretty much corresponds to loading dependencies.

@time using MyPackage
10.520202 seconds (14.28 M allocations: 822.814 MiB, 3.81% gc time)

@time using Images, ImageIO, Distributions
9.792210 seconds (14.22 M allocations: 819.221 MiB, 2.34% gc time)

To further reduce latency I believe I have to check out PackageCompiler.jl or rewrite code without external Packages.

1 Like
#8

Try upgrading to 1.6-beta. For me, on 1.5

julia> @time using Images, ImageIO, Distributions
  9.281217 seconds (19.22 M allocations: 1.062 GiB, 4.74% gc time)

but on 1.6

julia> @time using Images, ImageIO, Distributions
  4.413289 seconds (7.16 M allocations: 511.739 MiB, 5.40% gc time, 23.08% compilation time)

Also, Images is a large umbrella package. If you need only a subset of functionality, start with ImageCore and then add other specific components as needed.

4 Likes