[ANN] StrucRev - revise type definitions as you go

The package is intended to enable redefining structs within Revise.jl based workflow. It exports the macro @strev which (temporarily) wraps your struct like this

@strev struct S0
    x
end

becomes

module StrucRev_S0
  struct S0
      x
  end
end
S0 = StrucRev_S0.S0

If you edit S0, you will get a warning like
WARNING: replacing module StrucRev_S0
the S0 variable in your module will be re-bound to this new StrucRev_S0, and you can continue to code.

In case you want to define outer constructor methods, the usage is like following

@strev begin
    struct S0
        x::Int
    end
    S0(x::Real) = S0(Int(round(x))
end

If your struct definition contains references to some external variables, they will be imported into the struct-wrapping module.

Another usage is

@strev const c = 1

which will rewrite it to

c::typeof(1) = 1

thus enabling you to experiment with the value of c (but not with its type).

After you are finalized your types & constants, you simply remove all occurences of @strev (and, well, re-start Julia).

For more usage examples see runtests.jl

The package is not registered yet, and can be installed from GitHub. I haven’t yer done any really thorough test with it.

I still have some “imposter syndrome doubts” :worried: - first, if it is that simple, why nobody implemented it yet (or maybe it is already done), and, second, I am definitely not good at macroprogramming and maybe very naive and made some terrible mistakes. If that is not the case and the package is of some value, I plan to add some documentation, more testing, CI, and then register it.

Thanks are due to @disberd for advice and a piece of code.

P.S. Sure there exist packages for the same purpose, which somehow escaped my attention till now - see discussion here

17 Likes

I’m probably getting something wrong but the module wrapping seems like how Pluto notebooks are implemented, where you can redefine structs. Of course this more directly wraps the struct instead of a cell, and can be used in non-Pluto notebooks.

2 Likes

@Benny is exactly right. This is how Pluto does it! It moves the entire notebook into a new module each time a struct changes and ports forward everything it can from the old module.

5 Likes

Very cool package!

I just want to suggest a name change if possible: StructRev instead of StrucRev

I’m also thinking about @rev instead of @strev but maybe this is more controversial :smiley: at the same time it plays better with the revision of constant variables, I’m not sure if there is some other macro out there already called @rev

3 Likes

Or yet still: StructRevise may be more idiomatic for Julia packages.

10 Likes

I may have missed something, but if I understand correctly what StructRev does, I would anticipate the following issue:

# This is the equivalent for @strev Foo IIUC
julia> module Module_Foo
       struct Foo
           x::Int
       end
       end
Main.Module_Foo

julia> Foo = Module_Foo.Foo
Main.Module_Foo.Foo

# Now we define a method working on `Foo`
julia> foo(a::Foo) = a.x
foo (generic function with 1 method)

# And we use it on a newly created instance
julia> a = Foo(42)
Main.Module_Foo.Foo(42)

# So far so good
julia> foo(a)
42

Now what happens when we try to change the definition of Foo:

# Again, this is the expansion of @strev Foo IIUC
julia> module Module_Foo
       struct Foo
           x::Int
           y::Int
       end
       end
WARNING: replacing module Module_Foo.
Main.Module_Foo

julia> Foo = Module_Foo.Foo
Main.Module_Foo.Foo

# We have to create a new instance (the old one is outdated, fair enough)
julia> b = Foo(42, 43)
Main.Module_Foo.Foo(42, 43)

# But the problem is, methods that have been defined to accept the old
# struct definition don't know about the new one.
#
# (And arguably the error message is confusing!)
julia> foo(b)
ERROR: MethodError: no method matching foo(::Main.Module_Foo.Foo)

Closest candidates are:
  foo(::Main.Module_Foo.Foo)
   @ Main REPL[3]:1

Does StrucRev have a way to deal with this?

As far as I understand, tools like Pluto avoid this by putting everything into a new module: the new struct definition, all method definitions and all uses of the structure.

4 Likes

Yes, you are right, that appears to be a problem.

It is currently possible to put all constructors into the new module, and it will be possible to implement putting everything else therein, but then it would be definitely much less elegant.

I once experimented with an idea along the lines of:

# file `mysrc.jl`

# This macro does nothing
# But when anything passed to the macro changes, `Revise` will have to
# revise everything. This is a (hacky?) way of forcing a bunch of
# definitions to be revised together.
macro revise_together(expr)
    esc(expr)
end

# We use it to group both the struct definition and the related methods
@revise_together begin
    struct Foo_v1
        x :: Int
    end
    Foo = Foo_v1

    foo(a::Foo) = a.x
end

Now, in a REPL:

julia> includet("mysrc.jl")

julia> a = Foo(42)
Foo_v1(42)

julia> foo(a)
42

If you now change the struct definition in the file to (in the real thing, the versioning would be done by an other macro):

    struct Foo_v2
        x :: Int
        y :: Int
    end
    Foo = Foo_v2

you can go on in the same REPL:

julia> b = Foo(42, 43)
Foo_v2(42, 43)

julia> foo(b)
42

julia> methods(foo)
# 1 method for generic function "foo" from Main:
 [1] foo(a::Foo_v2)
     @ /tmp/mysrc.jl:12

I actually can’t remember why I didn’t explore this further. Of course there is the issue that it’s not always practical to group all methods pertaining to a struct near the struct definition. But I’m kind of confusely remembering that there was a more fundamental reason why the effectivity of this technique was limited, but I just can’t remember exactly why.

2 Likes

Now compared my approach with the “competition” - the packages RedefStructs.jl and ProtoStructs.jl .

RedefStructs.jl appears to be broken under Julia 1.10 , at least it errored on a minimal struct definition.

ProtoStructs.jl worked, however it suffers from the same problem as described above by @ffevotte *.

My package appears on the first glance to have all the abilities of ProtoStructs.jl (it supports @kwdef BTW), and provides the possibility to (re-)define inner and outer constructors, which appear to lack in the above package.

Thus it is indeed worth to register my package in its current form (which I was not sure). Will be done after adding some documentation.

The future plan is to enable co-wrapping of related methods. In the limiting case wrapping of “everything”.

@Tortar, @Alec_Loudenback - bikeshedding is welcome, and naming is difficult. For the macro name, I’d stay with the current one. For the package name, I’m not sure yet, but StructRevise could indeed be a better choice.

*EDIT: It is a different problem, see below.

While I think your approach is worthwhile it is not true that ProtoStructs.jl suffers from the same problems:

julia> using ProtoStructs

julia> @proto struct Foo
                  x::Int
              end

julia> foo(a::Foo) = a.x
foo (generic function with 1 method)

julia> a = Foo(42)
Foo(42)

julia> foo(a)
42

julia> @proto struct Foo
                  x::Int
                  y::Int
              end

julia> b = Foo(42, 43)
Foo(42, 43)

julia> foo(b)
42

It could be a different problem, but with an “includet” file I got an error

Error: Failed to revise ...
│   exception =
│    AssertionError: ld[idx] < typemax(eltype(ld))

Right now it’s midnight here, tomorrow I’d provide more details.

yes workflows using Revise don’t play well at the moment with ProtoStructs, but this is not a fundamental limitation as far as I can tell, more than nobody has made a PR fixing it yet

Reported the issue on the GitHub

made a PR trying to fix the compatibilty problem: Create revisableproto macro for compatibility with Revise by Tortar · Pull Request #35 · BeastyBlacksmith/ProtoStructs.jl · GitHub if someone is interested/would like to propose a better solution

1 Like

Is it somehow possible do detect whether the code runs under Revise or outside of it? There should be a way, I think.

BTW I’ve done a short test of your PR in a “Package under Revise” - both re-definition of structs and the of dependent methods appear to function OK.

1 Like

yes, it could be maybe possible, I asked a question about it Is there a way to know if `Revise` is enabled?