What's the best practice to dev a package with pluto and Revise when i need to change struct very often?

General Usage
,
1

The main issue is i need to change the definition of struct very frequently when developing my package, but revise cannot deal with that, which is a known limitation of Revise.jl.
So everytime I changed something with my struct, i have to restart julia, ]activate . , using Pluto, Pluto.run(), which is pretty an annoying process. I also did some reading in discourse, and it seems that there isn’t a good solution for that yet, maybe i missed something?

But I did some experinment with Pluto, and I found that, if I define some struct in a cell, and add some methods to that struct, I can actually change the struct definition in the original cell, and the struct is properly changed, without any issue, no restart needed. Even const variable in cell can be deleted and “redefined”.
But I cannot delete the “using MyPkg” cell, and add it back, to make it reload properly. there will be errors showing: “MethodError: no method matching MyStruct”.

So I have a strange idea, if pluto could add a devimport(“MyPkg”) function(whatever name is ok), which is equivelent from copy and paste things from MyPkg into a lot of hidden cells, so everytime i change the struct definition, it’s like changing content in a cell, and everything just works.
This doesn’t need to change anything of the language itself, or add scaffolding macros to the code which should be deleted later, or keeps renaming struct names.
Seem like a good workaround for package developing for me…

Or could revise use the mechenism behind pluto to make struct also reviseble?

So is this idea possible?
If it’s not possible, what’s your practice when you need to change struct definition when developing a package?
I need to use pluto because my package testing involve a lot of visualization and interaction, i cannot do it in repl.
Or if everything above failed, how can i clean my pluto environment within my pluto notebook without restarting everything?(I tried using the stop button and restart, that doesn’t fix the module struct redifinition problem) I heard about workspace or reload function in old version of julia but not anymore.

2

With Pluto you don’t need Revise at all.

But it’s not really clear what you’re asking. Please make a MWE (minimal example).

1 Like
3

I’m developing MyPkg

in MyPkg → src → MyPkg.jl

module MyPkg
export MyStruct, doit
struct MyStruct
x::Int
end
function doit(x::MyStruct) = x.x
end

in notebook.jl

#cell 1
using Revise

# cell 2
using MyPkg

#cell 3
a = MyStruct(20)

#cell 4
doit(a)

if I don’t do “using Revise”, anything i change in MyPkg.jl won’t have any effect.
if I use revise, like above, most change would effect except when i change struct def.
for example if i change MyStruct into:

struct MyStruct
    x::Int
    a::Int
end

then, everything with MyStruct won’t work, and revise would broke:

Failed to revise MyPkg\src\MyPkg.jl
exception =
│    invalid redefinition of constant MyStruct
│    Stacktrace:
│     [1] top-level scope
│       @ ~\Project\MyPkg\src\MyPkg.jl:5
1 Like
4

What I meant was that when you are developing your code, just have it in a Pluto notebook and not in a package. Then you can change the structs as you like.

Once you are happy with how the structs behave, you can put the code into a package.

1 Like
5

but what i want to dev is quite big, involving a lot of files, step by step. In the begining, one field would do, later on when there is need, i want to add some field, and see how everything interact with each other.
in my real case, i’m building a model for Neuron, in the begining i have only voltage field, and many function to model voltage related dynamics, but later on when this is stable, i want to add more states like dopamine and others. and the function that deals with voltage would now deal with dopamine and voltage, forming a more complicated dynamics. And during such development, i would constantly running simulation and plots, interactively, try out different parameters and different way of implement dopamine mechenism, which might involving changing def of strucct.
I shouldn’t copy paste everything in my big package into pluto every time right?

1 Like
6

You can make your methods definition more flexible so they “work” with many different structs (You can use an Abstract Type). Then, you only need to develop the struct in Pluto and use the package methods.

7

methods dispached for concrete types would be needed afterall, what can i do for those?

8

Have a look at https://github.com/BeastyBlacksmith/ProtoStructs.jl (or just use NamedTuples).

2 Likes
9

Maybe my case is a bit special, in my current design, i have Neurons{T}, which describe the property of all kinds of neurons, with the assumption that different types of neurons are different for their output method, but react the same for any kind of input.(Neurons receive input from other neurons)
So for Neurons{:Inhibitory}, it’s output would inhibite the target neuron. Changing the voltage field of the target neuron.
But if I want to add Neurons{:Dopamine}, not only i need a specific output function dispatch on it, I also need add a new state for the Neurons{T} struct, because all neurons would be affected by dopamine the same way by changing the new state.

This is just an example, there will be a lot more state changing for neurons for different neuronal mechenism.

10

this don’t work with constructors with new() in it right?
also don’t work with abstract type and parametric type, which are the 3 features i use right now…

11

3x no. I think either you eliminate the reliance on those 3 features during development and use NamedTuples, or you do restarts.

12

the most simple workaround i can think of is add a S::Dict field, and init with empty Dict, and work on that, a bit slower but ok for dev, and eventually change keys in dict back to fields for performance…
also i can have a getter function to deal with fields and keys of dict, to even simplify this workaround…

1 Like
13

I found with these help function, i can basically add and use more fields on the fly, without losing much.

struct MyStruct <: WhateverAbstractType
    x::Int
    S::Dict{Symbol, Any}
    function MyStruct(x::Int)
        new(x, Dict())
    end
end

# init specific property
function initproperty(obj::MyStruct, ::Val{:SomeState})
    return rand(obj.N)
end

function Base.getproperty(obj::MyStruct, sym::Symbol)
    if hasproperty(obj, sym)
        return getfield(obj, sym)
    else
        if !haskey(obj.S, sym)
            # init a property when first used.
            obj.S[sym] = initproperty(obj, Val(sym))
        end
        return obj.S[sym]
    end
end

# use this if your struct is mutable
function Base.setproperty!(obj::MyStruct, sym::Symbol, value)
    if hasproperty(obj, sym)
        return setfield!(obj, sym, value)
    else
        obj.S[sym] = value
    end
end

Is it possible to make this workaround more general? maybe write a macro? i’m not very good at macro…

1 Like
14

So what I found out that works quite well for redefining structs in files with Pluto is the following.

Have a Pluto cell with this
https://github.com/fonsp/Pluto.jl/issues/115#issuecomment-661722426

And then import your methods as

X = ingredients("../src/MyPkg.jl")
import .X.MyPkg = MyStruct

If you were now to change your source files, manually rerun the above cell and your method / struct definitions will be reevaluated.

It’s not as reactive as Revise.jl (meaning you to manually run the cell / import the methods again) but it gets the job done.

1 Like
15

Thanks for the solution! but I think you meant

X = ingredients("../src/MyPkg.jl")
import .X.MyPkg: MyStruct