How can the weight of package dependencies be determined?

General Usage
,
1

I often hear people talking about ‘heavy’ and ‘light’ package dependencies.
How is this determined?
Is the weight of each package determined by its size within the folder: “.julia/compiled/v1.11” ?

For example: 
                       Makie 0.24.6 ... 659.1 MB    
                       Plots 1.41.1 ... 614.4 MB
                   DataFrames 1.8.0 ... 426.3 MB
                  CairoMakie 0.15.6 ... 420.2 MB
                     GLMakie 0.13.6 ... 256.5 MB
                     Unitful 1.25.0 ... 214.2 MB
                PrettyTables 3.0.10 ... 174.4 MB
                       HTTP 1.10.17 ... 166.3 MB
                        CSV 0.10.15 ... 164.0 MB
                       Revise 3.9.0 ... 159.2 MB
                          Gtk 1.3.1 ... 135.1 MB
                  PythonCall 0.9.28 ... 133.1 MB
                        FFTW 1.10.0 ... 126.5 MB
                StaticArrays 1.9.15 ... 111.2 MB
                      Colors 0.13.1 ... 78.4 MB
                ColorSchemes 3.31.0 ... 62.2 MB
             Distributions 0.25.120 ... 51.2 MB
                        HDF5 0.17.2 ... 41.5 MB
                         GR 0.73.17 ... 39.6 MB
                  ForwardDiff 1.2.1 ... 35.8 MB
                   PythonPlot 1.0.6 ... 34.2 MB
                   TensorCast 0.4.9 ... 34.0 MB
                   StatsBase 0.34.6 ... 28.5 MB
                          DSP 0.8.4 ... 28.3 MB
                    PGFPlotsX 1.6.2 ... 24.5 MB
                TerminalPager 0.6.4 ... 20.6 MB
                        JSON 0.21.4 ... 17.3 MB
              Interpolations 0.16.2 ... 16.4 MB
                   PlotlyJS 0.18.17 ... 16.3 MB
                  RecipesBase 1.3.4 ... 14.9 MB
                   LazyArrays 2.6.3 ... 14.6 MB
                   TimeZones 1.22.0 ... 13.7 MB
                   Accessors 0.1.42 ... 11.9 MB
                         Proj 1.9.0 ... 11.7 MB
                      Impute 0.6.12 ... 9.3 MB
                    CondaPkg 0.2.33 ... 9.3 MB
               BenchmarkTools 1.6.0 ... 8.7 MB
                      Tables 1.12.1 ... 8.4 MB
                      Crayons 4.1.1 ... 8.3 MB
                    StatsFuns 1.5.0 ... 8.1 MB
                  FillArrays 1.14.0 ... 7.0 MB
                    NanoDates 1.0.3 ... 6.1 MB
                        XLSX 0.10.4 ... 5.5 MB
                 StructArrays 0.7.1 ... 5.2 MB
                  StatsPlots 0.15.8 ... 4.9 MB
                       Format 1.3.7 ... 4.6 MB
          KernelInterpolation 0.3.5 ... 4.5 MB
                    InspectDR 0.4.6 ... 4.2 MB
               DataFramesMeta 0.7.1 ... 3.4 MB
                      QuadGK 2.11.2 ... 3.3 MB
                   BonitoBook 0.1.0 ... 2.8 MB
                      Dierckx 0.5.4 ... 2.2 MB
                   IterTools 1.10.0 ... 1.9 MB
                IntervalSets 0.7.11 ... 1.8 MB
             StringDistances 0.11.3 ... 1.7 MB
                Combinatorics 1.0.3 ... 1.6 MB
    ConvolutionInterpolations 0.1.4 ... 1.5 MB
                  Parameters 0.12.3 ... 1.4 MB
                    LazyStack 0.1.3 ... 1.1 MB
            SplitApplyCombine 1.2.3 ... 0.9 MB
             ConstructionBase 1.6.0 ... 0.9 MB
             OrdinaryDiffEq 6.102.1 ... 0.8 MB
                     Measures 0.3.2 ... 0.7 MB
       ScatteredInterpolation 0.3.6 ... 0.7 MB
            FastRunningMedian 0.3.1 ... 0.7 MB
              ProgressLogging 0.1.5 ... 0.6 MB
            DataManipulation 0.1.21 ... 0.6 MB
                         Glob 1.3.1 ... 0.5 MB
                      LsqFit 0.15.1 ... 0.5 MB
                 DateFormats 0.1.19 ... 0.5 MB
               FastChebInterp 1.2.0 ... 0.5 MB
                 LaTeXStrings 1.4.0 ... 0.5 MB
                  PrettyPrint 0.2.0 ... 0.4 MB
                        Chain 1.0.0 ... 0.4 MB
           GridInterpolations 1.3.0 ... 0.2 MB
             AccessibleModels 0.1.8 ... 0.1 MB
2

I’d determine the “weight” of a dependency mostly by how many transitive dependencies it introduces, and second, by how long the package’s precompilation takes. That’s probably somewhat correlated with its size in .julia/compiled, but that’ wouldn’t be my primary metric.

3 Likes
3

and maybe the import/using time

2 Likes
4

The previous answers focus mostly on performance and TTFX but there are other aspects to bringing in more dependencies (direct or indirect), such as

  • Vulnerability to package deprecation or lack of maintenance
  • Maintenance burden when your dependencies release breaking updates

For instance, Ivet Galabova gave a talk at JuliaCon Local Paris where she explained that HiGHS (a state-of-the-art optimization solver) has no dependencies whatsoever, not even BLAS.

5 Likes
5

Add still the invalidations brought in by the dependencies. In my view there aren’t really light weight dependencies. GMT.jl has currently only 2 direct .jl dependencies (PrecompileTools & Tables). All others are stdlibs or _jll. The _jll artifacts may also have their own issues with DLL dependencies but are instantaneous to load and don’t compile to tens/hundreds MB cache files.

6

Thanks for the helpful comments.
I would have hoped there was an easy way to determine the “weight” of a dependency.

7

There are ways to measure the various metrics we talked about, the hard part is deciding which one(s) you care about and why

1 Like
8

For instance, consider the “using/import load time” metric for the dependencies in some user package.

The @time_imports macro, as suggested in this other post, appears to be a valuable resource. However, it outputs a substantial number of items from unknown origins. This can be overwhelming for beginners.

I was just thinking: would it be too much to ask for just one metric, like the body mass index?

9

The in-development version of the Performance tips page in the Julia Manual has an Execution latency, package loading and package precompiling time section.

1 Like