[ANN] ExplicitImports.jl: tooling to help use and maintain explicit imports in your package

Package Announcements
20

ExplicitImports v1.13

ExplicitImports v1.13 now requires Julia 1.10+. This makes ExplicitImports a bit easier to maintain and allows us to use newer Julia features in the package itself.

ExplicitImports v1.13 also now supports ignoring parts of source code or submodules: https://juliatesting.github.io/ExplicitImports.jl/stable/api/#Advanced:-telling-ExplicitImports-to-ignore-parts-of-source-code. I expect this to be a niche feature.

More importantly, it also drops JuliaSyntax and AbstractTrees as Pkg-managed dependencies. JuliaSyntax in particular was presenting some compatibility issues; ExplicitImports v1.12 updated to JuliaSyntax v1, but some other packages (or versions of packages) do not support JuliaSyntax v1 leading to compatibility conflicts. This is particularly problematic for ExplicitImports for two reasons:

  • ExplicitImports needs to load the user’s package in the same session as it performs a dynamic (not static) analysis. So the ExplicitImports CLI app, for example, would fail to analyze a package which (perhaps transitively) required a different JuliaSyntax than ExplicitImports itself. I have not seen reports of this in practice yet, likely since JuliaSyntax is a somewhat rare dependency and the CLI app (which probably comes with higher expectations of freedom-from-compat-issues than the library) is not widely used.
  • ExplicitImports and other dev dependencies like Cthulhu.jl which depend on JuliaSyntax are often added to the global environment or in startup files. Once one of these libraries loads some version of JuliaSyntax, the package is loaded once and for all. Then if the user activates an environment (perhaps even a temp env) and tries to load a package which requires a different JuliaSyntax version, it will “silently” break. Pkg will not report a compatibility violation (as each environment is separately valid) but the loaded version of the package will be incompatible with the required version of the package. There have been 3 reports of this.

ExplicitImports v1.13 solves these issues by vendoring JuliaSyntax and AbstractTrees. I.e. the source code of JuliaSyntax and AbstractTrees is programmatically copied into the source code of ExplicitImports during development so that ExplicitImports has submodules corresponding to particular versions of JuliaSyntax and AbstractTrees which are independent of the packages themselves. This allows us to remove the dependencies from our Project.toml to allow Pkg to freely load whichever versions of the libraries the user needs, since we will rely on our own private copies of the code instead. ExplicitImports now only has Pkg dependencies on Markdown, Pkg, PrecompileTools, and TOML.

Vendoring is not a general solution to compatibility problems, although it works well in this case. If you are thinking about vendoring a dependency, here are some things to consider:

  • a private copy of a module means types and functions are not shared. ExplicitImports.Vendored.JuliaSyntax.SyntaxData is a different type than JuliaSyntax.SyntaxData. If other libraries define methods for types or functions of JuliaSyntax, we won’t see them, since they won’t apply to our private copy. This totally breaks composability and multiple dispatch! That’s fine for our use case here, but can be a big issue in other use cases.
  • one no longer gets patch releases “for free”; if a vendored dependency needs an update, the library that vendored it needs to run a script to update the code, then release its own patch release.
  • we no longer have the power of Pkg to manage dependency versions, e.g. to choose a different version of the dependency depending on the Julia version. This makes it hard to vendor dependencies which are tightly coupled to Julia itself, like PrecompileTools.
  • all transitive dependencies either need to become our own dependencies or be additionally vendored. In the case of ExplicitImports, neither JuliaSyntax nor AbstractTrees add additional transitive dependencies, making this easy for us.
9 Likes