Bisecting regressions across dependencies

General Usage
1

I have this medium-sized industrial model written in ModelingToolkit, optimized via Optimization.jl. When I run it (warm) in my environment from January 22nd, it completes in less than a second. However, after updating my environment this morning, it now takes minutes to run, and hits MAXITERS (so it’s not just generic slowness; algorithmically it’s doing something different).

There’s a specific question here of “does anyone have any idea what happened in the ecosystem between Jan22 and today, that might be the culprit?”

But the broader question is how to deal with this kind of issue. If this was a problem with a “normal” package, I would either produce an MWE, or bisect that package’s commit/version to find the problematic commit. However:

  • Slimming our code down to an MWE is always an hours-long ordeal. Regular Julia code can be divide-and-conquered easily, but MTK models have complex interactions that make this very messy (and the JIT cost of each change is 30s+).
  • MTK+Optimization.jl have hundreds of dependencies. How, how, how am I supposed to bisect the problematic package/commit, if I don’t even know which package is at fault? Precompilation alone takes 5minutes+ on my laptop.

Are there tools that people use to help? I’ve often wished I could Pkg.update(; at_date=Date(2026,1,22)) and resolve to all the versions at the specific date. That way I could surely bisect the package update that broke my code. But it does not seem like that functionality exists?

What do people do?

2

You actually can, sort of: set JULIA_PKG_SERVER to the empty string, that’ll force to use the General registry as a git repository, and you can manually check out the revisions you want (you’ll find it in .julia/registries/General). Cloning the repo the first time will take a while.

1 Like
3

Cool suggestion! I love that this will be git bisect compatible!

But isn’t the docs at odds with your description?

Specifies the URL of the package registry to use. By default, Pkg uses https://pkg.julialang.org to fetch Julia packages. In addition, you can disable the use of the PkgServer protocol, and instead access the packages directly from their hosts (GitHub, GitLab, etc.) by setting: export JULIA_PKG_SERVER=""

I’m happy to use the PkgServer protocol for downloading packages; I’d just like to bisect the registry. Copilot suggests

Pkg.Registry.add(Pkg.RegistrySpec(path="/full/path/to/MyGeneral"))

which sounds good; I can have that path be my git cloned version, and bisect that.

4

No, how does that contradict what I said? Disabling the PkgServer has as a side effect switching to the registry as a git repository. Packages are still installed regularly (just by using the git protocol instead of going though the PkgServer). And you don’t have to do this forever, can just be only temporarily in one session of your terminal.

5

I don’t think any pkgserver manipulations are needed at all. Just remove General.toml file, and do registry add git@github.com:JuliaRegistries/General.