# \[201908\] Compiling a Data Science Image for Julia on Windows to combat compilation latency
**URL:** https://discourse.julialang.org/t/201908-compiling-a-data-science-image-for-julia-on-windows-to-combat-compilation-latency/27640
**Category:** General Usage
**Created:** [August 17, 2019, 1:09am UTC](https://discourse.julialang.org/t/201908-compiling-a-data-science-image-for-julia-on-windows-to-combat-compilation-latency/27640 "2019-08-17T01:09:00Z")
**Posts on this page:** 7
**Page:** 1
### Author:  [@xiaodai](https://discourse.julialang.org/u/xiaodai)
#### Post date: [August 17, 2019, 1:09am UTC](https://discourse.julialang.org/t/201908-compiling-a-data-science-image-for-julia-on-windows-to-combat-compilation-latency/27640/1 "2019-08-17T01:09:00Z")
I wanted to share some steps on how to compile a Data Science Image for Julia.
# 1. Remove existing installation of PackageCompiler.jl
Firstly, if you have PackageCompiler.jl previously, make you remove them entirely
- In Julia `]rm PackageCompiler`
- Manually remove the PackageCompiler folder from `C:\Users\UserName\.julia\dev\PackageCompiler`
- Manually remove the PackageCompiler folder from `C:\Users\UserName\.julia\packages\PackageCompiler`
- Manually remove C:\Users\RTX2080.julia\compiled\v1.1\PackageCompiler
- Close down Julia
# 2. Install PackageCompiler
use the `sd-notomls` branch of PackageCompiler
```julia
]up; add PackageCompiler#sd-notomls
```
# 3. Install all Data Science packages!
```julia
]add GeometryTypes PooledArrays Arrow Tracker ScikitLearnBase
add IteratorInterfaceExtensions SparseArrays VisualRegressionTests DelimitedFiles
add Images Dates LaTeXStrings ImageMagick DataStructures BinaryProvider
add CodecZlib DataValues Statistics UnicodePlots Distances CategoricalArrays
add RDatasets Random Test FileIO LinearAlgebra
add Tables TableTraits WeakRefStrings FlatBuffers Parameters
using PackageCompiler
pkgs = [
:Plots
,:GR
,:StatsPlots
,:DataFrames
,:DataFramesMeta
,:StatsBase
,:GLM
,:Clustering
,:Flux
,:Lazy
,:Feather
,:DecisionTree
,:CSV
#,:Query
#,:TableReader
]
@time sysnew, sysold = PackageCompiler.compile_incremental(pkgs..., install = true)
```
# 4. replace the sys.dll with the compiled one
Now go to `C:\Users\USERNAME\.julia\packages\PackageCompiler` and there should a sub-folder with in some random string code e.g. `sTrwT`, go inside in to find the `sysimg` folder.
Now copy the `sys.dll` to `C:\Users\USERNAME\AppData\Local\Julia-1.1.1\lib\julia`. Take care to back up the sys.dll to be overwritten if you wish.
# 5. Use it
Close down Julia and open it again and now you should notice that the packages load instantly!
# Notes
On my machine it took about 20mins to compile all the packages. Even after compiling there is a still a first to plot problem because `Plots.jl`’s test doesn’t have many `plot(...)` tests and PackageCompiler just compiles the tests inside it. Looking for tips on how to overcome this! Thanks!
---
### Author:  [@jlperla](https://discourse.julialang.org/u/jlperla)
#### Post date: [August 17, 2019, 1:16am UTC](https://discourse.julialang.org/t/201908-compiling-a-data-science-image-for-julia-on-windows-to-combat-compilation-latency/27640/2 "2019-08-17T01:16:22Z")
Is the `#notomls` branch still necessary? If so, is there any way for people to agree on what needs to be done for us to merge stuff we need onto master/tag on a release? @arnavsood @sdanisch?
---
### Author:  [@xiaodai](https://discourse.julialang.org/u/xiaodai)
#### Post date: [August 17, 2019, 1:18am UTC](https://discourse.julialang.org/t/201908-compiling-a-data-science-image-for-julia-on-windows-to-combat-compilation-latency/27640/3 "2019-08-17T01:18:13Z")
From my testing, that branch is necessary, or else it will fail.
---
### Author:  [@xiaodai](https://discourse.julialang.org/u/xiaodai)
#### Post date: [August 20, 2019, 7:08am UTC](https://discourse.julialang.org/t/201908-compiling-a-data-science-image-for-julia-on-windows-to-combat-compilation-latency/27640/4 "2019-08-20T07:08:20Z")
Seems to be working on v1.2 as well. Did some editing.
Edit:
Actually, the compile DLL doesn’t work. When I start Julia with the dll it crashes right away.
---
### Author:  [@arnavsood](https://discourse.julialang.org/u/arnavsood)
#### Post date: [August 23, 2019, 10:25pm UTC](https://discourse.julialang.org/t/201908-compiling-a-data-science-image-for-julia-on-windows-to-combat-compilation-latency/27640/5 "2019-08-23T22:25:44Z")
I don’t think the `sd-notomls` branch is necessary for the reason we needed it before (grabbing test deps, so that we could manually fudge things.) And I can get things to AOT-compile without using
it by using `master`.
If you have a look at (for ex.) some of his recent [NextJournal images](https://nextjournal.com/julia-environments/plots-environment?change-id=CQRybQ9RsNgda3xXeTctiC&node-id=67c3a2b3-e6c7-4863-962b-4034731a1726), they also aren’t using it… so dunno.
---
### Author:  [@xiaodai](https://discourse.julialang.org/u/xiaodai)
#### Post date: [August 24, 2019, 12:11am UTC](https://discourse.julialang.org/t/201908-compiling-a-data-science-image-for-julia-on-windows-to-combat-compilation-latency/27640/6 "2019-08-24T00:11:58Z")
If I use master, this is the errro I get. But may be I shouldn’t use `install=true`?
```julia
[ Info: used 3677 out of 3679 precompile statements
ERROR: MethodError: no method matching compile_incremental(::String, ::String; install=true)
Closest candidates are:
compile_incremental(::Union{Nothing, String}, ::String; force, verbose, debug, cc_flags) at C:\Users\RTX2080\.julia\packages\PackageCompiler\h11nw\src\incremental.jl:134 got unsupported keyword argument "install"
Stacktrace:
[1] kwerr(::NamedTuple{(:install,),Tuple{Bool}}, ::Function, ::String, ::String) at .\error.jl:125
[2] (::getfield(PackageCompiler, Symbol("#kw##compile_incremental")))(::NamedTuple{(:install,),Tuple{Bool}}, ::typeof(compile_incremental), ::String, ::String) at .\none:0
[3] #compile_incremental#63(::Array{Any,1}, ::Base.Iterators.Pairs{Symbol,Bool,Tuple{Symbol},NamedTuple{(:install,),Tuple{Bool}}}, ::Function, ::Symbol, ::Symbol, ::Vararg{Symbol,N} where N) at C:\Users\RTX2080\.julia\packages\PackageCompiler\h11nw\src\incremental.jl:177
[4] (::getfield(PackageCompiler, Symbol("#kw##compile_incremental")))(::NamedTuple{(:install,),Tuple{Bool}}, ::typeof(compile_incremental), ::Symbol, ::Symbol, ::Symbol, ::Vararg{Symbol,N} where N) at .\none:0
[5] top-level scope at util.jl:156
```
---
### Author:  [@fabgrei](https://discourse.julialang.org/u/fabgrei)
#### Post date: [August 24, 2019, 11:39am UTC](https://discourse.julialang.org/t/201908-compiling-a-data-science-image-for-julia-on-windows-to-combat-compilation-latency/27640/7 "2019-08-24T11:39:20Z")
you linked to old images (for julia 1.0). The newer images (julia 1.2) use #notomls. See a recent nextjournal post here. Sorry for the missing link. I am on my phone.
EDIT:
recent discourse post: [Julia 1.2 in Nextjournal](https://discourse.julialang.org/t/julia-1-2-in-nextjournal/27801)
current images: [https://nextjournal.com/julia](https://nextjournal.com/julia)