My guess is that widespread reevaluation is feasible because Pluto notebooks and cells don’t get very large, and this doesn’t hold if we replaced cells with modules and notebooks with packages.
1 Like
That’s fair. This is a similar problem to running the Debugger in Interpreter Mode only. Julia doesn’t have the facilities for middle ground performance (at the moment at least), but much like the Debugger problem is solved by opting certain (most) modules into Compiled Mode, you could opt into const-ing modules and only track the ones in development.
Trying out Pluto via the browser demo link, so bear with my inexperience. I had to even learn that editing the text multiple cells doesn’t immediately change the reactive code in them, as indicated by a yellow highlight of the cell. Anyway, Pluto can actually highlight a disadvantage of automatic code reevaluation:
# cell 1
struct X end
# cell 2, x = X()
x = X()
## change to
# cell 1
struct X a::Int end
# cell 2, MethodError
x = X()
When you change an intended call signature, you also need to change the calls in the rest of the code for it to adjust. There’s no general sensible way to replace the calls before automatic reevaluation. The MWE is simpler, you could deem that all X() calls should be replaced with X(1) calls by default, and you can patch those changes later. But in more complicated cases, there won’t be a 1-to-1 default replacement. When automatic reevaluation ventures into calls in the global scope, it will hit these MethodErrors, possibly after wasting some expensive computation beforehand. I suppose this is why includet doesn’t try to reevaluate global calls even after you manually fixed the call signature.
julia> write("baz.jl", "baz() = 1; a = baz()");
julia> using Revise
julia> includet("baz.jl")
julia> baz(), a
(1, 1)
julia> write("baz.jl", "baz(x) = 3; a = baz(2)");
julia> baz(2), a
(3, 1)
julia> baz()
ERROR: MethodError: no method matching baz()
1 Like
I think an essential element of a “Smalltalk Mode” would be also emulating the “always in the Debugger” aspect.
You would never hit a Method Error. You’d come to a breakpoint, be asked to define a solution (a new method for X() or redefining the function that calls it), and then rerun the frame with the solution in place.
To be fair, I’m mostly speaking in ignorance here – I haven’t used Smalltalk more than superficially, and it seems that its totally pure OO model is inextricable from the development loop is facilitates. But it definitely has me thinking!
1 Like
Am I reading this right to think that you can hit a method error if you rerun the code with no changes or improper changes to the calls’ signatures? Or do you mean that the debugger always stops you until you fix it?
1 Like
I think it would be something like
- Hit errant call site
- Open debugger panel that shows the error, the stack trace, etc. and has controls for moving up and down the call stack to inspect things
- Open a REPL/editor where you can enter a new method/redefine a method
- Rerun from a frame as far up the call stack as you’d like, calling
@invoke_lateston everything to pull in your changes. - Proceed until returning to top level OR hit error and go back to (2)
You’d never actually have any errors thrown, just identified, and you’d be equipped to fix them in real-time.
1 Like
Cool that you are trying Smalltalk. Always liked it’s minimalistic approach to OOP.
Regarding the debugger, you are right that Smalltalk – as well as Common Lisp for that matter – do not throw an error, but instead just invoke the debugger on top of the existing stack frame. Then, you usually have different interactive options on how to continue, i.e., fix it right away, just take a new value for now and continue, which are called restarts in Common Lisp. The crucial difference to most other languages with try-catch type error systems is that the stack is not unwound on hitting an error, i.e., throwing you down to the top-level or the place of the catch call. This gives the handling code more options as it can still access the original stack frame where it can possible fix the error.
Both in Smalltalk and Common Lisp, the default handler just opens the debugger and allows to fix the error interactively. In principle, you could do all your development in Smalltalk from the debugger, i.e., write a test → run it → get the debugger as no method is defined yet → write the method there → continue → get the debugger again (either as the fix was not complete or just on the next test). Probably invented before already, but apparently TDD was quite natural in Smalltalk.
3 Likes
Does Smalltalk immediately try to reevaluate things and hit the debugger upon a redefinition, though? Editing all the call sites I know of would be more convenient to do before attempting reevaluation, which hits an error and, if I understand it correctly, lets me make edits anyway. That’s why I was more partial to Common Lisp editing instances of redefined classes earlier, I thought it dodged the issue of manual call site fixes needing to happen first.
This problem is not different from the case when you redefine a struct and then have to restart the REPL – if there is an error on the call site then someone needs to fix it, regardless, so just throw an error like Revise does it.
I would not try to patch the caller’s code automatically, because 1) as you say it is ambiguous and 2) that would also entail to change source code on disk.
Why not just add an option one can set to control if global code should be re-evaluate or not? Revise.jl already has that for includet.
Naively, I would say that reevaluating the global state is what the default should be (in the order files were loaded), because after all you want to avoid the cost of restarting a session which will also reevaluate the global state. Unless the magic of replacing all occurrences of the old struct and invalidation is significantly more costly, then this should be cheaper then restarting – isn’t this the goal?
Of course, reevaluating global state only makes sense if correctness can be ensured. I know there are these artificial (or real-world?) examples where a open call writes some data to a file. Rerunning global state would then write more than needed.
I would guess there is no universal solution to handle all those cases (otherwise someone already would have implemented it). So my guess would be to come up with workarounds for such situations (perhaps a @macro annotation that blocks global reevaluation), but tell people they should only use it if necessary. This would go hand in hand with the “general wisdom” of using globals only if really needed.
In the case the latter is not possible for a developer, then he/she should turn off the global-state reevaluation or go back to restarting the REPL.
I know, lots of guessing here from my side, so please ignore if I seem to be out of touch with the actual difficulties of the implementation 
EDIT: Fix typos.
2 Likes
Well, only evals the changes. If you continue from my example earlier
julia> __revise_mode__ = :eval;
julia> write("baz.jl", "baz(x) = 4; a = baz(2)"); # change method, not call
julia> baz(2), a
(4, 1)
julia> write("baz.jl", "baz(x) = 4; a = baz(5)"); # change call
julia> baz(2), a
(4, 4)
And that involves the issues of the select evaluations not occurring in the same order as a full restart’s evaluations, which can easily change behavior. Imagine if I had added a b = a+1 when a lagged in the change, then changed a without changing b.
Sorry, should have been more clear about Revise: I know it does not function the way I described it above atm, but I think it contains all the necessary tools to track global statements such that it can re-evaluate all of them after anything changed in that file.
Assuming it would work that way, your new MWE should just print print 4 for a in both cases since a = ...; b = a + 1 are both global statements and will need reevaluation once baz is changed, or not?
So, for Smalltalk, EVERYTHING is an object. So “redefining a Struct” is actually “sending a message to a Class Object requesting that it change its fields.” But to do this, the Class Object sends a message to every instance object it has created requesting that they update their fields. And when that message gets sent out, the first instance will raise the Debugger complaining that it doesn’t have a “update fields to blah blah” method. And the programmer then writes that method and all the other instances are updated with that method so their fields are auto-updated. It’s also the case that, because there is no multiple dispatch, every name refers to a specific, concrete object. So the Refactor window will take you to an exhaustive list of where the class object is reference and help you update everything.
This is why it doesn’t cleanly map onto Julia. It is decidedly not object oriented. And we can refer to symbols that are not yet defined. We have a lot more flexibility but that means we’d need to approach tracking the relationship between value, symbol, and types differently.
3 Likes
Just to add on this:
-
On changing a class, nothing is really reevaluated – in the sense of rerunning previous source code. Instead, as @mrufsvold nicely explained as method is send to the class to update itself. In turn, all instances are updated to be consistent with the new class layout. This process can be customized by suitable methods, but might leave some new fields uninitialized by default.
-
Updating objects mainly keeps the image, i.e., the state of all currently existing objects (which in Smalltalk really includes everything, i.e., windows, buttons, application state etc) consistent. It does not ensure that the resulting state would be the one obtained by rerunning the source code.
There is no such notion of source code in Smalltalk by the way, i.e., the code of every method is stored in its corresponding class in some internal AST-type respresentation which is itself an object. Also constructors are just a method to the class itself, in turn stored in its metaclass. To the user/programmer code is only shown in a textual representation when viewed in the browser, but not actually stored in that way. I.e., when changing anything – methods, objects etc. – think of it as a refactoring which might require calling code. Smalltalk has great tools for that as all its code is represented by objects and finding dependencies such as callers or callees is just a method away, but it will not do any such change for you.
Programming in a Smalltalk image is really very different from a REPL in many other languages. Common Lisp might come close, but most REPLs merely feel like stupid shells in comparison.
2 Likes
AFAIK it’s not typical for a class to track all its instances in OO languages, though. I wonder if Smalltalk’s classes actually hold references to its instances or use the garbage collector to check all objects for the class’s instances. It also sounds like it’s not reevaluating code so much as converting instances 1-to-1, so I’m seeing a similarity with CLOS as described earlier in the thread.
I meant that reevaluation upon changes would have a different order from that of a full restart, and Revise doesn’t track and eval dependent unchanged lines like Pluto does unchanged cells. Continuing from my 1st example, ignoring the 2nd:
julia> __revise_mode__ = :eval;
julia> write("baz.jl", "baz(x) = 4; a = baz(2); b = a+1"); # edit baz, add b
julia> baz(2), a, b # a is unchanged, b used obsolete a
(4, 1, 2)
julia> write("baz.jl", "baz(x) = 4; a = baz(2)+0; b = a+1"); # edit a
julia> baz(2), a, b # obsolete b
(4, 4, 2)
I fear we are talking about different things.
I meant that reevaluation upon changes would have a different order from that of a full restart, …
I think we agree that we don’t want this.
I do understand that this is how __revise_mode__ = :eval behaves at the moment.
What I was suggesting is that when it comes to implementing support for updating structs, then one should record the order in which global statements (I guess I should just say variables?) were defined when a file was included. If that information is available when a struct is redefined, then after doing all the invalidation magic etc. one just replays/reevaluates the recorded statements in their recorded order (provided their order was not modified too).
So it should behave as if one would include the file again.
Wouldn’t this guarantee correctness?
1 Like
Many lines would have to be reevaluated, not just lines that call the type constructor or use its instance (x = X(); a = x + 1; b = f(a) would all need to be reevaluated, and b = f(a) does not call X or use an instance of X). That could add up to such a large portion of the source code, especially in packages or files including multiple other files, that it doesn’t save time with the added overhead of invalidation and reevaluation tracking, let alone be fast enough for interactivity. This is also assuming that reevaluation doesn’t have invalidated calls. Those need to be fixed manually before reevaluation can work. Based on the few languages that do it, it really looks like replacing live instances rather than reevaluating lines to replicate the source code is the way to go for interactivity.
I would argue that it would be at least as fast as having to restart the REPL, assuming the invalidation process is cheaper than whatever needs to be done when loading code in a fresh session. This would already better than what we now have.
Many lines would have to be reevaluated, …
Yes. And I can see that this is problematic for Jupyter notebooks etc. where you usually rely on cached results which could have taken minutes to compute.
But I think the situation is different for package code: To this day I have not yet seen a Julia package that does a minute long computation on every startup to cache some result that is then needed for actually using the package. I would even go so far as to say that 95% of all global variable computations are not what make package loading in julia slow. Someone please correct me if I am wrong.
Obviously, I am just guessing at this point.
Personally, I would rather just lean back for a few seconds and let all the globals being recomputed than being dropped in a debug mode where I have to walk though 10s or more global variables and being ask if I want to reevaluate them or not. In the worst case I forgot to reevaluate one and then I have to undo and redo the structure change to get it to the desired state.
Lastly, from my programming experience (including other languages), most of the time you edit functions and not structs. So the question is how much latency can a user endure when it comes to this non-trivial task of struct redefinition? Apparently, having to restart the REPL is already too costly. Will rerunning global statements be also too costly? Will be being prompted into the debug mode be cheap enough?
This is what I’m not assuming, in fact tracking and identifying code to reevaluate for each type will take up time and memory, like how methods make backedges to implement invalidations. That overhead would have to be there even if you don’t end up redefining any types personally.
People try to make as few global variables as possible, but that doesn’t mean global variables have little influence. A type doesn’t just show up as the occasional global instance cache, it shows up in methods as annotations or global references. And when a global instance shows up, it may be a temporary value that determines other persistent global instances or conditionally defines methods. Method invalidations were already a large and actively addressed problem for package loading latency because compiled code was being thrown out; type redefinition would do much more at minimum, and even more if you try to evaluate other lines to replicate a restart. The other thing is that invalidated methods are only recompiled on demand, so it helps interactivity. If you want to replicate a restart, you can’t run select lines on demand, you must run the necessary subset in order at once (see the last __revise_mode__ = :eval; example for what happens if you rerun lines out of order).
Well, you might have a point there. I give up and continue restarting my REPL.