Workflow challenges with redefining structs (from "Why I still recommend Julia")

Community
1

"Hence, the development workflow is to start the process, get a cup of coffee while it loads for the first time and keep the process alive for the rest of the day". – Or take another cup of coffee each time you change structure definition… :laughing:

7 Likes
2

There’s proto structs package and revise for that right?

3 Likes
3

Hmm, how many times do I change the definition of the structure? If I keep changing the structure repeatedly, I have probably not thought this through properly…

5 Likes
4

Yes, and you definitely need more coffee to think thoroughly!

3 Likes
5

I found myself often adding more fields to an existing Struct or adding parametric types as I proceed. At the very beginning, it may be hard to coin a perfect struct. I prefer starting coding as soon as I get the idea and polishing later as I go.

9 Likes
6

Another cause of restarts is when some function becomes unresponsive for a long time and ignores ctrl+c. This happens to me often.

9 Likes
7

Pluto is great for this: change functions/structs/whatever, it’s all automatically kept up-to-date.
When the code has somewhat stabilized, put it into a proper module/package outsides of the notebook. Struct changes tend to be much more rare afterwards.

7 Likes
8

Just develop in a module and reload the module.

9

Really wish that when people suggested this, they would be more specific about the way and the limitations. Modules, types, and macros aren’t as dynamic as a function’s method table is now, and though Revise.jl gets you farther, you need to do more work and can only go so far. I’ve seen a fair share of posts being confused why they “reloaded” a module but all the dependent modules still use the old version.

There really should be a citable central source of how different things reference each other in Julia, e.g. how does f know what MyType is in f(x::MyType) = _f(x), with the express goal of teaching people to reason what is and isn’t affected when code is reevaluated. Obviously it should state the caveat that some information are only implementation details.

1 Like
10

I agree - I learned by experimenting. maybe there is docs maybe not - but there should be.

11

What do you mean by “develop in a module” and “reload the module”?

1 Like
12

When you reload or re-evaluate the module the type can change. This is good enough for me most of the time but as Benny pointed out it doesn’t solve all problems

13

I mean how do you do that? Sorry for the stupid question.

14

Do your bleeding-edge development in a project, where you have a file like

module WIP
struct DraftType
 # etc
end
function foo(x::DraftType); do_stuff(x); end

# maybe include("other_bits.jl")
end

Now from the REPL (i.e. the Main module), do
include("wip.jl")
Experiment with WIP.DraftType and WIP.foo. When you find mistakes or deficiencies, edit and include over again. You need to remember to recreate any DraftType objects. Do not do using WIP in the REPL, to avoid being haunted by ghosts.

17 Likes
15

Wow, I do not know this before. Thanks! But I feel this does not fit my workflow well. I have a relatively large package to maintain and sometimes I need to update my struct definition. I have to experiment with several ways to update the struct to find which one is better.

1 Like
16

You can take it to the extreme of

include(joinpath(ENV["HOME"] ,".julia/dev/MyPackage/src/MyPackage.jl"))

so your large package is an ephemeral module for the duration. I’ve found this to be productive, but it does take some extra thought and effort.

17

You can do this in other simple ways. Create a module structs at the top and define all your structs inside of it. Experiment with the structs as you wish and modify structs inside the module. When you are finished, remove the module Structs and its end statements as well as all Structs. prefixes.

module Structs
struct Point
    x::Float64
    y::Float64
end
end

Structs.Point(1.0,2.0)
 # Main.Structs.Point(1.0, 2.0)

module Structs
struct Point
    x::Float64
    y::Float64
    z::Float64
end
end

Structs.Point(1.0,2.0,3.0)
 # Main.Structs.Point(1.0, 2.0, 3.0)
1 Like
19

In this workflow, one would also need to redefine functions that accept WIP.DraftType:

julia> module WIP
       struct DraftType
        # etc
       end
       function foo(x::DraftType); do_stuff(x); end

       # maybe include("other_bits.jl")
       end
Main.WIP

julia> bar(::WIP.DraftType) = 2
bar (generic function with 1 method)

julia> bar(WIP.DraftType())
2

julia> module WIP
       struct DraftType
        # etc
       end
       function foo(x::DraftType); do_stuff(x); end

       # maybe include("other_bits.jl")
       end
WARNING: replacing module WIP.
Main.WIP

julia> bar(WIP.DraftType())
ERROR: MethodError: no method matching bar(::Main.WIP.DraftType)
Closest candidates are:
  bar(::Main.WIP.DraftType) at REPL[2]:1
Stacktrace:
 [1] top-level scope
   @ REPL[4]:1

julia> methods(bar)
# 1 method for generic function "bar":
[1] bar(::Main.WIP.DraftType) in Main at REPL[2]:1

julia> bar(::WIP.DraftType) = 2
bar (generic function with 2 methods)

julia> bar(WIP.DraftType())
2

I must say that the output of methods is confusing, and wish that there was a way to indicate ghosting.

4 Likes
20

I‘ve never used it myself, but Stefan Karpinski said somewhere that using NamedTuples as structs during development is also possible and then swap them out by structs once you‘re done

2 Likes
21

I haven’t tried this but I can totally see how this would work a lot of the time.