[ANN] Higher productivity (fewer Julia restarts) with Revise.jl

Community
, , ,
21

i wonder how many memory leaks in julia will be discovered if restarting occurs less often :slight_smile:

22

A brief update here. Many platform and IDE-specific issues have been fixed, thanks mostly to users who have reported bugs and contributed fixes. In particular, if you use VS Code’s console it’s worth trying the initialization here discovered (I believe) by Kristoffer Carlson and David Anthoff.

For users of the in-development Julia 0.7, I just merged some fairly large changes (not yet tagged, but for the adventurous available on the master branch):

  • Older versions of Revise needed to be able to parse your source code to determine which files you were includeing in your package. On 0.7, this is no longer required. In theory, Revise should now work for any package (if not, please report as a bug).
  • Hacking on Base is now easier and more fun. Just say Revise.track(Base), and Revise will attempt to incorporate any differences between the current source files and their state when you last built Julia.
  • Revise should be faster and use less memory. For precompiled packages, due to changes in 0.7 it only needs to parse the source code when one of the files changes. This should reduce or eliminate the performance hit when loading packages.

Revise still has significant limitations (it can’t handle type definition changes, method deletion, and a few other kinds of changes), but I’m optimistic that at least some of these will be resolved in the fullness of time.

25 Likes
23

Revise is amazing. It feels like magic.

8 Likes
24

I realize that this needs changes to the language, not just Revise.jl, but addressing these two issues would mean that I would only have to restart very rarely. This is another way to mitigate the problem of compilation times, hope that if it does not get addressed before v1.0 then it is possible to do it some backward-compatible way so we don’t have to wait until v2.0.

Thanks for the great work, this library improved my workflow so much that it is hard to even quantify.

25

I have felt that the need to regularly restart the REPL has been something of an elephant in the room and that it was an impediment to the marketing of the language. (It make me think of how you might try to market MATLAB or Visual Studio if you had to regularly shut it down and then restart it for your code changes to take effect).

As far as I am concerned Revise is a game changer! Thank you for your work. I do hope it makes its way into Base. It makes introducing the language to new users so much easier.

2 Likes
26

I hope it never makes it into Base. That would mean 3-6 months between updates, as opposed to getting them every few days or weeks like we do now, with just a Pkg.update().

27

But once things have stabilized you don’t need to update it so often…

28

Code may stabilize regarding bugs, but I hope it doesn’t “stabilize” when it comes to features. Also, as Julia becomes more stable, its release frequency should decrease too, exacerbating the problem.

The fewer things in Base, the better. That said, distributions of Julia may just install a curated set of packages, so it comes with “batteries included”. It is my perception that things are moving in this direction anyway.

1 Like
29

I agree with this in general, but the features of Revise.jl are so fundamental that I find it hard to agree that they should not somehow be baked into the core of the language.

Edit: or at least be available by default.

2 Likes
30

I agree with @peterkovesi that this should be unintrusive for the end user. It would be weird (for a newbie) if you needed to load a package and set it up to get this behavior.
If that package is loaded and set up automatically then it doesn’t matter of course whether it is in base or it is a package that comes with the default installation. All is fine as long as a newbie doesn’t need to worry about it and things “just work”.

And indeed a very big “thank you” to Tim Holy for creating this!

31

I realize that this needs changes to the language, not just Revise.jl, but addressing these two issues would mean that I would only have to restart very rarely.

Agreed that these are currently major barriers. Personally, I’d guess the method deletion issue counts for something like 80% of my restarts. Here there is some hope: there is already a stub of a solution, though it’s pretty far from being viable. Assuming we don’t make the necessary changes to Base in time for 1.0, this would be a new feature that I’m guessing can be done in a backward-compatible way, so presumably could appear in 1.x. The timing will depend a lot on how hard the Base changes are, and I confess to not really having a clue about that at the moment. Fortunately, Jameson has noticed the issue, and if you have Jameson thinking about the problem, who knows what kind of astonishing wizardry might someday be possible?

4 Likes
32

Thanks for the kind remarks, everyone!

I agree with the general sentiment that it would be nice to make discovery of Revise and related packages more straightforward. My only real idea is to have Pkg.init offer to install some core “user experience” packages such as Gallium (now that it seems to be coming back) and Revise. (Maybe plotting too? IDEs? Where does one draw the line?) Then we’d just need a good build script in Revise. One potential hitch is that build scripts seemingly can’t prompt users with questions, because they run in a separate process, so we can’t ask users whether they want to have Revise start up automatically or only when they say using Revise in their current session.

@StefanKarpinski, as the Pkg3 guru, any thoughts?

1 Like
33

Pkg3 is already far more interactive and we can certainly add more smarts to that over time. Suppressing interactivity when not wanted is definitely necessary as well, and I guess we’ll figure that out. It’s still a work in progress.

2 Likes
34

Is it possible for Revise.jl to also update generated code when the code generation is modified in a pkg?

35

This issue https://github.com/timholy/Revise.jl/issues/31 ?

You can manually call revise(module_name) to force reevaluating every function in a module. Use it on the package where the generated function is defined. Similarly, if you update a macro, call revise on the modules that use the macro.

2 Likes
36

Out of curiosity, what’s the current plan regarding Revise.jl and Julia 0.7/1.0? Any chance Revise will be included in the “default” experience somehow?

37

It would be nice if it is built-in somehow, but only if it doesn’t increase the startup time significantly. Otherwise, I would rather use it optionally when needed.

38

FWIW I really hope it’ll become part of the basic user experience.

1 Like
39

Just to clarify, does “part of the basic used experience” mean

  1. built into Base or the standard library, or

  2. this, and also enabled by default

in this context? As much I love Revise.jl, I think of it as part of the IDE, not the language. Or is the problem is better discoverability?

40

The default initialization used by the REPL and Juno, perhaps? But two thoughts:

  • As irritating as the restarts may be for introduction, I suspect it is pretty low on the irritation list for introductory users (as opposed to package developers). But maybe it is also very low effort to add and test
  • A related consideration in all of this should be the alternative, Revise-free workflow that @ChrisRackauckas has championed: Using Juno for Interactive Test-Driven Julia Package Development - YouTube After learning about that, I didn’t end up needed Revise… but maybe that is because I am not working on fancy libraries.