Hello,
I’m starting off in Julia and I want to write a program to use a couple of Python libraries that I have. I have a script that I run non-interactively (I use emacs for dev) and it looks like this
cat test1.jl @time using PyCall: pyimport @time using DataFrames
and running it
julia test1.jl
3.237148 seconds (3.49 M allocations: 172.424 MiB, 1.63% gc time)
3.796700 seconds (3.47 M allocations: 183.872 MiB, 0.64% gc time)
So every time I run it, I have to wait 7-8 seconds before it runs any of the rest of my code. So my question is simple - can I reduce that time? Do I have to precompile it into my own module, or something?
I am using Julia 1.4.2 on Debian 10. I haven’t set up Conda or anything - am just letting Julia sort it out as it seemed I don’t need Conda in Linux (but should I?)
OK, I now see that modules get pre compiled. So I tried a separate module that loads PyCall etc, but that separate module doesn’t get compiled, but I do see PyCall in the compiled directory. So I guess maybe this is as fast as it gets. I must be missing something
You’re not missing anything, its just that the developement workflow of repeatedly launching a non-interactive script is fairly incompatible with Julia at the moment exactly because of these long loading times. The workflow that works much better is to instead launch Julia once, and keep that session going as long as you can.
For example, launch Julia then call include("test1.jl"), then maybe make some edits to the script in emacs, then call include("test1.jl") again, etc…, rather than launching a fresh session each time.
The package Revise is also pretty indispensable and can automatically reload code in scripts or in modules when it changes. Both Juno and julia-vscode have features to help develop your code while keeping a session going as long as you can. Ultimately there’s also PackageCompiler but I would exhaust the other options before trying this more advanced / experimental one.
(You will find tons of discussion on long loading times on this forum, sometimes under the name “time to first plot”)
This delay is also expected to improve with Julia 1.5 (which is currently in beta and will be released soon) and then to improve even more with Julia 1.6. If you want more details, check out: Update on latency and the typocalypse