Adding MORE packages can result in speedup (and other strangeness)

General Usage
1

A.
Speedup and lower memory use, adding the first package:

$ julia -q
julia> @time using DataValues, CSVFiles, ExcelFiles, StatFiles, ParquetFiles, DataFrames, VegaLite
 10.102168 seconds (14.07 M allocations: 842.677 MiB, 3.38% gc time)

vs:
$ julia -q
julia> @time using CSVFiles, ExcelFiles, StatFiles, ParquetFiles, DataFrames, VegaLite
 10.522314 seconds (14.33 M allocations: 857.825 MiB, 3.36% gc time)

$ julia -q
julia> @time using CSVFiles, ExcelFiles, StatFiles, ParquetFiles, DataFrames, VegaLite
 10.528817 seconds (14.33 M allocations: 857.912 MiB, 3.54% gc time)

julia> versioninfo()
Julia Version 1.6.0-DEV.735
Commit f047d7ffc7 (2020-08-27 00:54 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Xeon(R) CPU D-1541 @ 2.10GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-9.0.1 (ORCJIT, broadwell)

Is this a strange bug? I’ve heard people shying away from adding packages, but the trend is clear. Julia scales well.

I would love to hear a good explanation (I know about invalidations, but they may hurt, also help?).

B.
The order of the packages can also matter for speed.

Opposite order of using the SAME packages can mean 20% fewer allocations, i.e. allocating 207 fewer MB (19%) and 1.8 sec. quicker (i.e. takes 21% off):

https://github.com/queryverse/Queryverse.jl/issues/43

3 Likes