Question about transitioning to a Revise-based workflow

General Usage
1

I am making the move from 0.6 to 0.7 and transitioning from a reload("MyModule"); include("myscript.jl") workflow to a Revise-based workflow as recommended in the deprecation warning for reload. I am using the setup described in the Using Revise by default section of the documentation.

I’ll try to demonstrate the issue I am having with a MWE module called MyMod that goes through three versions and it seems like Revise falls a version behind and can’t get me up to the latest version of MyMod. Here’s what I see from within julia (version 0.7.0) as I go through this:

julia> import MyMod #importing version 1 of MyMod
[ Info: Precompiling MyMod [39b22132-abcb-11e8-1d53-e5fa610b8d45]

julia> revise(MyMod) #calling revise after going to version 2 of MyMod
true#note that "asdf" is not printed as I would anticipate for version 2

julia> revise(MyMod) #calling revise after going to version 3 of MyMod
asdf #now it prints "asdf"!
true #however, it doesn't print "blah" as I would anticipate for version 3 -- it seems like I am at version 2

julia> revise(MyMod) #calling revise again with the code still at version 3 of MyMod
true #neither "asdf" nor "blah" isn't printed -- still doesn't appear as if version 3 has been loaded

Here are the different versions of MyMod:

Version 1:

module MyMod
macro mymacro()
	:(1)
end
@mymacro
end

Version 2:

module MyMod
macro mymacro()
	println("asdf")
	:(1)
end
@mymacro
end

Version 3:

module MyMod
macro mymacro()
	println("asdf")
	println("blah")
	:(1)
end
@mymacro
end

The only way I can see to get to version 3 with a Revise-based workflow is by starting a new julia session. With the reload-based workflow I could have just reloaded and not had to restart julia. Given that the purpose of Revise is to reduce the number of times you have to restart julia (and given Tim Holy’s godlike julia prowess), I feel as if I am missing something or doing something wrong.

Is there a way to get to version 3 with a Revise-based workflow without restarting julia? With julia 0.7, is there a simple way that I can be confident that the latest version of my module has been loaded aside from restarting julia?

1 Like
2

Why do you need to call revise explicitly? I thought that it hooks into import and then just watches the files for changes, so if you are following the default setup, everything should work automatically.

3

From the docs:

If you change a macro definition or methods that get called by @generated functions outside their quote block, these changes will not be propagated to functions that have already evaluated the macro or generated function.

You may explicitly call revise(MyModule) to force reevaluating every definition in module MyModule . Note that when a macro changes, you have to revise all of the modules that use it.

I would file an issue with Revise.jl.

4

Yeah, that second part you quoted from the documentation was what led me to start calling revise(MyMod).

I also thought of revise(MyMod) as a replacement for reload("MyMod"). That is, it ostensibly provides an explicit mechanism (which I love) to get latest version of a module rather than an implicit mechanism (which I merely like).

I’ll open an issue at the Revise repo.

5

I’ll open an issue at the Revise repo.

I happened to notice this and beat you to it. Indeed there’s already a fix posted at Store unexpanded macros in defs (fixes #174) by timholy · Pull Request #175 · timholy/Revise.jl · GitHub.

To explain, let me at least show what was and wasn’t happening. This was my live session:

julia> push!(LOAD_PATH, "/tmp/pkgs")
4-element Array{String,1}:
 "@"        
 "@v#.#"    
 "@stdlib"  
 "/tmp/pkgs"

shell> cp MyMod1.jl MyMod.jl

julia> import MyMod
[ Info: Recompiling stale cache file /home/tim/.julia/compiled/v1.0/MyMod.ji for MyMod [top-level]

julia> MyMod.@mymacro
1

shell> cp MyMod2.jl MyMod.jl

julia> revise(MyMod)
true

julia> MyMod.@mymacro
asdf
1

shell> cp MyMod3.jl MyMod.jl

julia> revise(MyMod)
asdf
true

julia> MyMod.@mymacro
asdf
blah
1

On balance I’d say things mostly look good, right? However, I’m guessing you’re puzzled about why @mymacro doesn’t run each time. Here’s some insight:

julia> fi = Revise.fileinfos["/tmp/pkgs/MyMod/src/MyMod.jl"]
Revise.FileInfo(OrderedCollections.OrderedDict(Main=>FMMaps(<0 expressions>, <0 signatures>),MyMod=>FMMaps(<1 expressions>, <0 signatures>)), "/home/tim/.julia/compiled/v1.0/MyMod.ji")

julia> fi.fm[MyMod].defmap
OrderedCollections.OrderedDict{Revise.RelocatableExpr,Union{Nothing, Tuple{Array{Any,1},Int64}, RelocatableExpr}} with 1 entry:
  :($(Expr(:macro, :(mymacro()), quote…                                                                   => nothing

julia> first(fi.fm[MyMod].defmap)
:($(Expr(:macro, :(mymacro()), quote
    println("asdf")
    println("blah")
    $(QuoteNode(1))
end))) => nothing

In brief, what this means is that Revise is only storing a single expression, the definition of the macro itself. I am guessing that you’re expecting it to re-evaluate the macro? Note that wouldn’t happen if you said using MyMod again, so in a sense this is OK.

However, this was related to a real bug. Let’s change this to a macro that performs a transformation on a method definition:

# MyMod1.jl
module MyMod
macro change(foodef)
    foodef.args[2].args[2] = 1
    esc(foodef)
end
@change foo(x) = 0
end

# MyMod2.jl
module MyMod
macro change(foodef)
    foodef.args[2].args[2] = 2
    esc(foodef)
end
@change foo(x) = 0
end

# MyMod3.jl
module MyMod
macro change(foodef)
    foodef.args[2].args[2] = 3
    esc(foodef)
end
@change foo(x) = 0
end

then we get something that is clearly a bug:

julia> push!(LOAD_PATH, "/tmp/pkgs")
4-element Array{String,1}:
 "@"        
 "@v#.#"    
 "@stdlib"  
 "/tmp/pkgs"

shell> cp MyMod1.jl MyMod.jl

julia> import MyMod
[ Info: Recompiling stale cache file /home/tim/.julia/compiled/v1.0/MyMod.ji for MyMod [top-level]

julia> MyMod.foo("hello")
1

shell> cp MyMod2.jl MyMod.jl

julia> revise(MyMod)
true

julia> MyMod.foo("hello")
1

shell> cp MyMod3.jl MyMod.jl

julia> revise(MyMod)
true

julia> MyMod.foo("hello")
2

and the reason is that the macro has been expanded:

julia> collect(fi.fm[MyMod].defmap)
2-element Array{Pair{Revise.RelocatableExpr,Union{Nothing, Tuple{Array{Any,1},Int64}, RelocatableExpr}},1}:
 :($(Expr(:macro, :(change(foodef)), quote
    (foodef.args[2]).args[2] = 2
    esc(foodef)
end))) => nothing                         
                                                           :(foo(x) = begin
          2
      end) => (Any[Tuple{typeof(foo),Any}], 0)

You can see the @change isn’t present in the last entry. That’s because Revise was expanding macros in order to parse the signature (necessary for the recent fix of Line numbers · Issue #51 · timholy/Revise.jl · GitHub, and now more obviously by Rebugger). But it should have been storing the unexpanded version of the expression, which is what the fix in Store unexpanded macros in defs (fixes #174) by timholy · Pull Request #175 · timholy/Revise.jl · GitHub does:

julia> collect(fi.fm[MyMod].defmap)
2-element Array{Pair{Revise.RelocatableExpr,Union{Nothing, Tuple{Array{Any,1},Int64}, RelocatableExpr}},1}:
 :($(Expr(:macro, :(change(foodef)), quote
    (foodef.args[2]).args[2] = 3
    esc(foodef)
end))) => nothing                         
                                           :(@change foo(x) = begin
              0
          end) => (Any[Tuple{typeof(foo),Any}], 0)
5 Likes
6

Thanks for the quick fix and detailed response. I really appreciate all your contributions to the julia ecosystem.