How to Make Precompiling Faster?

[Freya_the_Goddess]

I add InteractiveDynamics and the precompilation of OrdinaryDiffEq took 5 hours. What are required to make the precompiling fast? high RAM or internet connection or processor power?

[baggepinnen]

For OrdinaryDiffEq in particular, you can ask it to not precompile the whole world using

using Preferences, UUIDs
set_preferences!(UUID("1dea7af3-3e70-54e6-95c3-0bf5283fa5ed"), "PrecompileNonStiff" => false, "PrecompileStiff" => false, "PrecompileAutoSwitch" => false, "PrecompileAutoSwitch" => false, "PrecompileDefaultSpecialize" => false, "PrecompileAutoSpecialize" => false, "PrecompileFunctionWrapperSpecialize" => false, "PrecompileNoSpecialize" => false)

see

https://docs.sciml.ai/DiffEqDocs/latest/features/low_dep/

[Benny]

5 hours does seem unusually long, though. Maybe it’d help to specify Julia version, OS, and maybe hardware specs? If you’re wondering about internet connection, maybe someone knows how Pkg.add times its downloads among the precompilation work.

[tim.holy]

If it’s 5 hours, there is something wrong. On my machine it’s typically under 5 minutes. Which is still long, of course, but on a different scale.

[johnh]

I timed @time Pkg.add(“InteractiveDynamics”) this morning. Agree with Tim Holy on the time. I can repeat and give exact time.
This is on a laptop running Fedora.
Are you very constrained in resources on your machine? Maybe low on rAM and swapping is happening?

[nilshg]

Although we have had other people on here before who had what initially looked like suspicious compile times where it then turned out they were on some really old single threaded i3 CPU or something, so maybe the ouptut of versioninfo() would help to understand what’s going on.

[johnh]

Please report versioningo() and your @time output

Also look at this thread

PackageCompiler compilation time > 2h

8Gbytes is not a lot of RAM these days but should be enough. I would advise openign a new terminal and running these utilities as the compilation is happening top htop iotop Often a simple ‘top’ will give you insight into what is going on.

[Freya_the_Goddess]

This is the versioninfo of Julia 1.9.2, processor is core i7, RAM is 16 GB.

Maybe the internet is very slow. I only get slow signal internet here. Furthermore, Maybe because I have added packages that requires a lot of power like InteractiveDynamics, DynamicalBilliards,…

I will use @time and share the time. Precompiling after long hours here usually fails.

[nilshg]

Hm, I’ve never seen that CPU bit a quick Google suggests it might be quite old?

What do you get when you precompile a smaller package, say DataFrames?

If you struggle with precompile times it might be good to switch to 1.10 alpha if you can, which reduces precompile times (albeit not by 5 hours unfortunately…)

[Freya_the_Goddess]

My laptop is using Custom Linux OS in QEMU to try this Julia

[johnh]

I would suggest opening a new ssh session to this VM and running top or htop as you run Julia.
top can be surprisingly useful in revealing problems.

How much RAM does this VM have allocated?

[Elrod]

@nilshg

Hm, I’ve never seen that CPU bit a quick Google suggests it might be quite old?

Very. Doesn’t even have AVX.
It is the second Intel Core processor. Latest release is the thirteenth generation.

Still, 5 hours sounds extreme. Like others noted, sounds like it could be swapping on a HDD.

[Freya_the_Goddess]

I will try your suggestion, I allocate 16 GB now. Fully.

[Freya_the_Goddess]

DataFrames is quick around 10 minutes, my laptop is not a sophisticated laptop, old Dell Precision.

[nilshg]

Okay in that case it does look like your system is just very slow, I just precompiled DataFrames on a 10 year old i5 system and it took a little over one minute.

[Freya_the_Goddess]

I just add SciMLTutorials and the time is 11342 seconds / 189.0333 minutes, I learn to be patience with this (in the main OS: CAELinux 2020 not inside Qemu, with the same Dell Precision Laptop Julia version 1.9.0)

[nilshg]

I think it’s a sad fact of life that on very underpowered hardware precompilation is a real pain, especially in 1.9. If you can I would use the recently released 1.10 alpha, which addresses some of the large increase in precompile times in 1.9.

[Benny]

@Freya_the_Goddess

high RAM or internet connection or processor power?

Just occurred to me, we can measure internet connection. When you Google “test internet connection”, it offers you a free test from the Measurement Lab. There is a privacy policy you should read to see if you’re okay with them collecting data, including your IP address. If you are okay with that, then it’s just hitting a button and letting a test run until it gives you 2 numbers. For example, when I run it on my machine and home internet, I get 379.4 Mbps download rate and 39.4 Mbps upload rate. We could check and compare to see if it’s a download speed issue.

[Freya_the_Goddess]

Yes I am downloading Julia-1.10 alpha.

I will use this new version for the Differential Equations package,
I add OrdinaryDiffEq 3 days ago, wait for hours, then there is this warning of NeuralPDE:
(wait for 28019 seconds or 466.983 minutesor around 7 hours )

Then anytime I add another package like DiffEqSensitivity it takes a long time to precompile OrdinaryDiffEq, in fact just now I try to add again OrdinaryDiffEq, it has to precompiling again, why?

This is the package I have downloaded for days I have to wait

Thanks for your patience and answers

[Freya_the_Goddess]

There are lots of ways to Rome, I click the Measurement Lab button, it hasn’t comes with result, so I just use this:

around 260 Kb/s, I am using slow internet connection unlike yours that can be tens or hundreds of Mb/s