Big updates in Revise-land

Package Announcements
1

I’m pleased to announce a series of Revise releases, most recently today’s 3.15.0, that come closer to fulfilling Revise’s Just Works goal. First let’s do the numbers: since v3.14.3 (released May 13), there have been:

  • 35 PRs merged across three releases
  • 38 issues closed, 36 of which predated this push. The oldest was filed in 2018, and the median had been open for about three and a half years.
  • Open issues dropped from 75 to 41, nearly half the backlog.

The remaining 41 are not being ignored; many are harder architectural questions, and a few are upstream Julia issues.

Big changes to file-watching and thread-safety

Perhaps the most important class of changes focus on event-handling:

  • End dropped filesystem events. Directory watching now uses a persistent, buffered monitor, so a change landing in the brief gap between checks is queued instead of lost. (This was rare, but with enough usage it does get hit occasionally.)
  • Windows filesystems under WSL. If you’re using WSL but working on source files that live on the Windows side, now Revise will track them (#514, from 2020). This is a brand new feature, implemented by automatically falling back to polling. While you will hopefully appreciate the expansion of what you can revise, note that the delay of polling might cause some initial confusion, particularly for agents (who might not wait very long before trying to re-execute code). If you work this way frequently, you may want to instruct them on the finer points.
  • Directories that vanish and reappear. A Pkg.build, a git checkout, or an editor’s atomic save can briefly remove a watched path. Previously this produced an alarming warning and permanently abandoned the watch; now Revise waits a grace period for the path to return (#523).
  • Symlinked .julia depots, stale source-text caches, friendlier guidance when you hit the inotify watch limit
  • Better aggregation by entr entr has long tried to combine events, but it should be much more robust now while holding locks for much less time.
  • Thread-safety. On Julia 1.12, package loading can complete on a background thread, which turned some long-latent races in Revise’s global state into live ones. All of that state is now behind a single lock. Formerly Revise used multiple locks to try to partition different activities (though not very well), but there are so many cross-overs that a single lock makes everything far more robust.

New capabilities

  • Packages baked into a system image. If you build a sysimage with PackageCompiler.jl, the package is already loaded at startup, so Revise’s usual load hooks never fire. Revise.track(ThePackage) now works for these, and it catches up on any edits made since the image was built (#685).
  • Late-added Distributed workers. Revise has long propagated revisions to existing workers, but a worker added afterward loaded stale code from disk and could fail when deserializing revised closures. New workers are now replayed the session’s revision history when they join (#637).
  • pkgversion stays current. On Julia 1.12+, bumping a package’s version in Project.toml while developing now updates the cached version that pkgversion (and Pkg’s compat checks) report, instead of requiring a restart (#684).

Julia 1.12+ changes

Julia 1.12+ allows Revise to update bindings. While the main target was struct revision (discussed below), there’s one nice change that is not related to structs: the “oops, I defined f(x::Foo) when I meant Base.f(x::Foo)” correction pattern now works. When the revision would have left a methodless local f shadowing Base.f, Revise detects the orphaned binding and repairs it (#239, filed in 2019). This new feature is on for everyone who uses Julia 1.12 or above; because Revise is making a guess that this is what you intended, you might see an @info informing you that the binding update happened.

Among recent changes, the biggest change to struct revision is that editing only a @kwdef default no longer triggers the expensive full-type-tree walk; Revise now recognizes when the type itself is unchanged (#1022).

However, struct revision is still off-by-default, as it can take a very long time. (Hopefully future Julia changes will allow this to be changed.) If you want to turn it on despite the costs, add a LocalPreferences.toml to your current project containing the contents

[Revise]
revise_structs = true

Latency

Loading Revise at startup (e.g. from startup.jl) no longer forces any Revise-owned compilation on Julia 1.10, 1.11, or 1.12: the last stragglers are now precompiled, and a --trace-compile test keeps it that way.

How this happened

Much of this work was done with AI assistance (you’ll see the co-author trailers in the commit log). As it turned out, the AI agents were significantly better than I am at starting from a vague “I tried changing my code and got this error:” and turning it into an actual minimal working example. This has always been the bottleneck for Revise development, as users often (understandably) don’t know the exact sequence of events that led to the error. In some cases, it’s possible that we didn’t identify the real cause, but landed on some other verifiable bug instead; if nothing else it’s an improvement, and a vague issue report isn’t very actionable, so I took the expedient route of assuming that any plausible MWE could, when fixed, claim credit for the issue.

In other cases, the issues were long-lingering due to lack of expertise on my part (I am not a filesystems or event-handling guru) or the simple fact that Revise development has always been a huge time sink because the issues usually involve subtle interactions touching deep fundamentals. It’s nice to be able to delegate much of the detective work.

Try it, and please report any problems

I am quite confident that the 3.15 release is the best Revise ever. Nevertheless, a huge wave of changes could cause problems. Please report these; I and my agents are standing at the ready to fix them.

21 Likes