Will Julia ever fix its "using ..." latency problems?

May I ask how long you’ve been around with julia? I certainly remember the days of 0.6, where e.g. using Plots took tens of minutes on the very machine I’m typing on now.

Also, there have been multiple links in this thread (and in the linked issues and other things) pointing towards improving that state even more. As a short recap what has happened in the past few months:

  • Deep investigations what made compilation slower than necessary
  • How to remove those unnecessary slow spots
  • Build better package serving infrastructure (PkgServer serving tarballs instead of downloading git repositories from GitHub…)
  • Figure out why some packages and code patterns took a long time to compile and fix those code paths in the compiler
  • Make the compiler itself faster
  • Investigate issues why certain packages increase load time dramatically
  • Fix those packages (this is the recent “crusade on invalidations”, where package code and functions have to be recompiled because a newly loaded packages caused a lot of already cached and compiled functions to have to be recompiled)

Some future work includes:

  • Cache not only lowered & typed code, but also native, compiled machine code
  • Cache more code between environments
  • Cache more code
  • Reduce compiler latency
  • maybe even serve compiled code?
  • Did I mention cache more code, even compiled code?

I’m only 95% sure that nothing similar would crop up, but yes, these specific problems are gone insofar you can’t express that idea at all anymore.

Without eval and @eval, there is no REPL and there is no interactive mode. There’s also no recompilation at runtime and there’s no shadowing of existing things in the same namespace since there’s no way to “redefine” anything like a struct or a function at runtime. There’s also no convenient creating of similar function expressions and writing boilerplate (where you define functions with different names but that can take the same arguments) would be a dread. Julia would just be another statically compiled language like C, C++ or Rust (or any number of other compiled languages).

It certainly wouldn’t be the end of the world and most of the time when people want to use eval, they really shouldn’t anyway (don’t parse & eval strings to emulate bad macros!). Their existence does make certain things much easier and provides a lot of flexibility though, so I’m really glad we have them. You also learn a lot about why it’s a bad idea to allow redefining everything at any time, so being limited to certain places makes the language much more powerful & performant as a whole.

Always remember, the really fast parts in Python are not written in Python :slight_smile:

9 Likes