Advice for dealing with `struct` during development

I find it quite annoying that I can’t replace a struct that I made. I read somewhere that I should put them in a module because a module can be replaced.

E.g.

module Distances
    struct KM{T <: Real}
        value::T
    end
end

Now if I run the below twice I get an error

import Main.Distances: KM

the error is

WARNING: ignoring conflicting import of Distances.KM into Main

Is there a way to allow me to import KM even after I’ve redefined it? I need this to iterate as I am in the design phase of many of my structs and I do tend to change their design quite regularly.

2 Likes

Have you seen Revise.jl? :slight_smile:

Edit: Revise.jl cannot handle changes to types like you’re asking about, but types tend to be more stable than methods, so still good to know about Revise.jl :slight_smile:

1 Like

Unfortunately, this is an outstanding issue. See

Some people recommend that you keep renaming said types, eg KM1, KM2, etc, and finalize when done. Personally, just restarting the process is less of a hassle for me.

6 Likes

I think this should work (I haven’t used it yet): use Revise.jl, prototype with NamedTuples, once +/- set turn them into a struct. You can use a constructor for the NamedTuple as provided by Parameters.jl: MyType = @with_kw (a=1, b, c=3), then later replace it with @with_kw struct MyType ....

Edit: doing dispatch on the NamedTuples will be not easy/possible though.

1 Like

That is what I do. I have configured my editor (emacs in my case, but this could probably be done in any serious editor / IDE) so that it performs this kind of search/replace for me. As long as you keep your type definition in one file, it is really easy to work in this way.

You can also define constructor functions with a non-numbered name so that client code can create instances of the correct type without having to know about the current iteration number.

Maybe an example is worth a thousand words:

module Bar

# Export a non-numbered name
export Foo

# Forward exported non-numbered functions / constructors to numbered ones
#
# Such definitions will disappear when the structure is stabilized and
# renamed "Foo".
Foo() = Foo_1()

# Bump this number when modifying the struct
struct Foo_1
    f::Int
end

# Update all uses in the relevant scope (file/module)
# this should be as simple as a find/replace
Foo_1() = Foo_1(0)

end
# External use:
# client code doesn't need to be adapted to the internal numbering in the
# module
julia> Bar.Foo()
# but you *see* that the structure has changed (useful to avoid
# hard-to-debug errors when different types have the same name)
Main.Bar.Foo_1(0)
2 Likes

Along the same lines, it occurred to me that with named tuples and getproperty, one could have

@flexible struct Foo
    x::Int
    y::Float64
end

expand into

struct Foo{NT}
    _vals::NamedTuple
end
Foo(x, y) = Foo((x=x, y=y))
getproperty(f::Foo) = ...

Then with Revise.jl one could change the fields of Foo as much as you want without restarting and strip off the @flexible once it’s “frozen”. It would have a few restrictions, but at least it would work in functions

f(::Foo, ...) = ...
12 Likes

I would not say so:

julia> TupleStruct = NamedTuple{(:a, :b), Tuple{Float64,Float64}}
NamedTuple{(:a, :b),Tuple{Float64,Float64}}

julia> subtract(x::TupleStruct) = x.a - x.b
subtract (generic function with 1 method)

julia> ts = (a=10.5, b=3.2)
(a = 10.5, b = 3.2)

julia> subtract(ts)
7.3

That’s what I do usually:

  1. Don’t import, just use Distances.KM in your test code (maybe with const D = Distances).

  2. Or, search and replace KM -> KM1, KM1 -> KM2, etc.

If you put your test code into another module (rather than Main), it will bind to the current version of the module you are working on every time you load it. So suppose DT.jl contains

module DistanceTests
using ..Distances # or Main.Distances, if always run there
function mytest(x)
  km = Distances.KM(x)
  useful_test(km)
end
end # module

then after you revise and reload Distances, just include("DT.jl") again and invoke the test functions.

My usual approach while prototyping is to put the new types in a temporary module inside a temporary file and just include that file multiple times while I sketch out the design. Once the sketch is done, the structures get copied into the package I’m working on. This is kind of ok, but awkward if the new structures need to interact with the rest of the package in a meaningful way.

Having said that, I like this idea of @flexible struct a lot more. It’s very simple, yet captures a lot of the behaviors you would want from this kind of thing.

2 Likes

What does the .. mean?

From the “Modules” section in the manual:

using ..Utils would look for Utils in Parent 's enclosing module rather than in Parent itself.

(implied context:

module Parent
using ..Utils
# something using bindings defined in Utils
end

)

The problem is not that you cannot alias a NamedTuple type and dispatch on it, but that you

  1. cannot distinguish this from another such type with the same layout,
  2. parametric types become tricky.

(Cf nominative vs structural type systems, Julia structs are nominative but the above solution is closer to structural).

2 Likes

I recently needed this and here it is https://github.com/BeastyBlacksmith/ProtoStructs.jl

5 Likes

I know this is an old thread, but I think the question not out-dated. I just stumbled across a blog with the following proposal

EDIT: I adapted the macro @kwredef to also include cases with typed structs.

macro redefinable(struct_def)
    struct_def isa Expr && struct_def.head == :struct || error("struct definition expected")
    if struct_def.args[2] isa Symbol
        name = struct_def.args[2]
        real_name = struct_def.args[2] = gensym(name)
    elseif struct_def.args[2].head == :<:
        name = struct_def.args[2].args[1]
        real_name = struct_def.args[2].args[1] = gensym(name)
    end
    esc(quote
        $struct_def
        $real_name.name.name = $(QuoteNode(name)) # fix the name
        $name = $real_name # this should be `const $name = $real_name`
    end)
end

This approach is somewhat similar to one of the approaches above in that it creates a true struct. The advantage is that the name is identical to the struct that you would normally define.
The only disadvantage could be that it creates a lot of structs and doesn’t check whether the new structure is identical to the old one. But for development purposes this shouldn’t be a problem. Also, the variable which holds the struct type could be overwritten.

I was about doing a similar thing for Base.@kwdef and borrowed some of this ideas:

macro kwredef(expr)
  expr = macroexpand(__module__, expr) # to expand @static
  expr isa Expr && expr.head === :struct || error("Invalid usage of @kwredef")
  expr = expr::Expr

  t = expr.args; n = 2
  if t[n] isa Expr && t[n].head === :<:
      t = t[n].args
      n = 1
  end
  curly = t[n] isa Expr && t[n].head === :curly
  if curly
      t = t[n].args
      n=1
  end

  T_old = t[n]
  t[n] = T_new = gensym(T_old)

  esc(quote
    Base.@kwdef $expr
    $T_old = $T_new
    $curly ? $T_new.body.name.name = $(QuoteNode(T_old)) : $T_new.name.name = $(QuoteNode(T_old)) # fix the name
  end)
end
1 Like

Brave souls could check out the branch of the upcoming Protostructs version: struct API by BeastyBlacksmith · Pull Request #4 · BeastyBlacksmith/ProtoStructs.jl · GitHub

Ideally all you’d need to do then is put @proto in front of the struct definition, and remove it once development is done.

Any feedback appreciated.

9 Likes

What do you think about also supporting Proto.@kwdef?
Youmight either use my above code example, which I adapted to also support typed structs, or you go back to the original definition of Base.@kwdef and insert your @proto macro in the the upper part.

Thats definitely a worthwile extension and might be added before the release

1 Like

Very recently I’ve put that blog post into a package and uploaded it to the registry. You can check it out here (and contribute new ideas!):

Edit: I added @kwredef to the latest RedefStructs.jl.

3 Likes

Okay, I merged that PR and triggered the registration of the new version, where keyword definitions should also work

1 Like