# What's the best practice in Julia for research (often changed codes)?

**URL:** <https://discourse.julialang.org/t/whats-the-best-practice-in-julia-for-research-often-changed-codes/58892>\
**Category:** General Usage\
**Tags:** design-pattern\
**Created:** [April 9, 2021, 12:18am UTC](https://discourse.julialang.org/t/whats-the-best-practice-in-julia-for-research-often-changed-codes/58892 "2021-04-09T00:18:01Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![iHany](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ihany/32/18151_2.png) [@iHany](https://discourse.julialang.org/u/iHany)\
**Post date:** [April 9, 2021, 12:18am UTC](https://discourse.julialang.org/t/whats-the-best-practice-in-julia-for-research-often-changed-codes/58892/1 "2021-04-09T00:18:01Z")

</div>

Hi, I’ve used Julia about one year and I still struggle with how to use it.

First of all, as a researcher, I often change my code including code structure and Julia types.  
When redefining types, Julia REPL needs to be restarted and I should spend quite a long time for precompilation.  
In my experience, the actual development speed is fast only for almost-completed codes.  
For some projects on progress, I always restart and precompile the whole code, which is so annoying.

What’s the best practice for often changed codes?

Note: I usually create my own package and add almost every functionality in the package for each project. Also, I use Revise.jl.

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [April 9, 2021, 12:55am UTC](https://discourse.julialang.org/t/whats-the-best-practice-in-julia-for-research-often-changed-codes/58892/2 "2021-04-09T00:55:01Z")

</div>

Some options to deal with the problem of Revise not being able to track type changes are discussed in some threads. For example:

> [@Workflow when experimenting with types](https://discourse.julialang.org/t/workflow-when-experimenting-with-types/56483):
>
> Hi all, longtime lurker and irregular Julia user here. Revise is absolutely great and I love it but I wonder if there’s a better workflow when experimenting with types. I try to get my types defined ASAP but especially in the beginning, when taking a stab at a particular problem I frequently find myself changing definitions. Currently I just restart the interactive in order to reload Revise and my type definitions but I feel like I’m doing something wrong or that there might be a better way. H…

---

<div class="post-metadata">

**Author:** ![iHany](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ihany/32/18151_2.png) [@iHany](https://discourse.julialang.org/u/iHany)\
**Post date:** [April 9, 2021, 1:29am UTC](https://discourse.julialang.org/t/whats-the-best-practice-in-julia-for-research-often-changed-codes/58892/3 "2021-04-09T01:29:41Z")

</div>

Well, I don’t think it is a cool solution.  
It sounds like, I have to replace all structs to NamedTuple for often-changed codes.

Of course, struct may not be changed frequently when the development becomes mature.  
Ironically, if the project becomes mature and huge, precompilation would take a long time when I realise that the struct should be changed.

EDIT: Apart from Revise, is it impossible to enable type redefinition in Julia in principle?  
It would be really great if someone tells me something about this functionality 🙂

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [April 9, 2021, 1:46am UTC](https://discourse.julialang.org/t/whats-the-best-practice-in-julia-for-research-often-changed-codes/58892/4 "2021-04-09T01:46:01Z")

</div>

Theoretically in the future, revise should be able to support this. The hard part is figuring out what to do with the old type. If you have variables that are of the old type, do you delete them? What about specialized functions?

---

<div class="post-metadata">

**Author:** ![iHany](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ihany/32/18151_2.png) [@iHany](https://discourse.julialang.org/u/iHany)\
**Post date:** [April 9, 2021, 1:51am UTC](https://discourse.julialang.org/t/whats-the-best-practice-in-julia-for-research-often-changed-codes/58892/5 "2021-04-09T01:51:02Z")

</div>

I don’t know how to “delete” certain objects. When I’ve searched it, many people recommended to just restart REPL, but my memory might be wrong.

If you don’t mind, can you describe what you meant in detail? For instance, with an example.

And I really hope that Revise once supports this functionality.

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [April 9, 2021, 1:53am UTC](https://discourse.julialang.org/t/whats-the-best-practice-in-julia-for-research-often-changed-codes/58892/6 "2021-04-09T01:53:29Z")

</div>

Here’s the basic problem. Suppose I define

```julia
struct Foo
    x::Float64
end
bar = Foo(1.0)
baz(x::Foo) = Foo(x.x -1.0)

```

If I now redefine `Foo` as

```julia
struct Foo
    y::Int
end

```

what happens when I try to call `baz(bar)`?

---

<div class="post-metadata">

**Author:** ![iHany](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ihany/32/18151_2.png) [@iHany](https://discourse.julialang.org/u/iHany)\
**Post date:** [April 9, 2021, 2:00am UTC](https://discourse.julialang.org/t/whats-the-best-practice-in-julia-for-research-often-changed-codes/58892/7 "2021-04-09T02:00:59Z")

</div>

Well, I’ve tried it but encountered the redefinition error.  
First, run the following (by `include("the_file_name.jl")`.

```julia
## Type redefinition test
struct Foo
    x::Float64
    # y::Int
end
bar = Foo(1.0)
baz(x::Foo) = Foo(x.x - 1.0)
@show baz(bar)

```

and changed the code to the following and re-run it.

```julia
## Type redefinition test
struct Foo
    x::Float64
    # y::Int
end
bar = Foo(1.0)
baz(x::Foo) = Foo(x.x - 1.0)
@show baz(bar)

```

It gives

```nohighlight
julia> include("test/tmp.jl")
ERROR: LoadError: invalid redefinition of constant Foo
Stacktrace:
 [1] top-level scope
   @ ~/.julia/dev/MyPackage/test/tmp.jl:2
 [2] include(fname::String)
   @ Base.MainInclude ./client.jl:444
 [3] top-level scope
   @ REPL[1]:1
in expression starting at /Users/jinrae/.julia/dev/MyPackage/test/tmp.jl:2

```

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [April 9, 2021, 2:02am UTC](https://discourse.julialang.org/t/whats-the-best-practice-in-julia-for-research-often-changed-codes/58892/8 "2021-04-09T02:02:27Z")

</div>

Sorry for the confusion, my point wasn’t describing something that currently works. It was about what would happen if redefinition were supported. Basically, it’s unclear how redefinition should behave, so it currently errors.

---

<div class="post-metadata">

**Author:** ![ettersi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ettersi/32/6829_2.png) [@ettersi](https://discourse.julialang.org/u/ettersi)\
**Post date:** [April 9, 2021, 2:51am UTC](https://discourse.julialang.org/t/whats-the-best-practice-in-julia-for-research-often-changed-codes/58892/9 "2021-04-09T02:51:22Z")

</div>

My approach:

- Don’t use types for “interactive” coding unless you really have to. I’ve come to appreciate that the types provided by `Base` / other people’s packages are sufficient much more often than one might think.
- If you do have to create your own types, put them into a module `MyModule` and don’t do `using MyModule`. Instead, either introduce an abbreviation `M = MyModule` and type `M.my_func()` while prototyping, or do something like in the following MWE:

```julia
julia> # Initial definition
       module MyModule
       export MyType
       struct MyType
           x::Int
       end
       end;

julia> module Foo
       using ..MyModule
       @show MyType(1)
       end;
MyType(1) = Main.MyModule.MyType(1)

julia> # Redefinition
       module MyModule
       export MyType
       struct MyType
           x::Float64
       end
       end;
WARNING: replacing module MyModule.

julia> module Foo
       using ..MyModule
       @show MyType(1)
       end;
WARNING: replacing module Foo.
MyType(1) = Main.MyModule.MyType(1.0) # Look ma, I changed Int to Float64!

```

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [April 9, 2021, 5:19am UTC](https://discourse.julialang.org/t/whats-the-best-practice-in-julia-for-research-often-changed-codes/58892/10 "2021-04-09T05:19:13Z")

</div>

Another option if you don’t like named tuples is to just call your struct `MyStruct_1` and if you need to change it, do a find replace on your code base to `Mystruct_2`.

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [April 9, 2021, 5:41am UTC](https://discourse.julialang.org/t/whats-the-best-practice-in-julia-for-research-often-changed-codes/58892/11 "2021-04-09T05:41:28Z")

</div>

One slightly simpler version of this is to use something like `MyStruct=MyStruct_1`, and then you can redefine what `struct` `MyStruct` is an alias of.

---

<div class="post-metadata">

**Author:** ![jzr](https://avatars.discourse-cdn.com/v4/letter/j/eb9ed0/32.png) [@jzr](https://discourse.julialang.org/u/jzr)\
**Post date:** [April 9, 2021, 5:48am UTC](https://discourse.julialang.org/t/whats-the-best-practice-in-julia-for-research-often-changed-codes/58892/12 "2021-04-09T05:48:39Z")

</div>

> [@nilshg](#):
>
> Another option if you don’t like named tuples is to just call your struct `MyStruct_1` and if you need to change it, do a find replace on your code base to `Mystruct_2` .

Could a macro do that?

```julia
@numbered_struct struct Foo
x::Int
y::Int
z::Int
end

abc = Foo(1,2,3)

```

expanding to

```julia
struct Foo_1
x::Int
y::Int
z::Int
end
Foo = Foo_1

abc = Foo(1,2,3)

```

---

<div class="post-metadata">

**Author:** ![lungben](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lungben/32/12314_2.png) [@lungben](https://discourse.julialang.org/u/lungben)\
**Post date:** [April 9, 2021, 6:18am UTC](https://discourse.julialang.org/t/whats-the-best-practice-in-julia-for-research-often-changed-codes/58892/13 "2021-04-09T06:18:06Z")

</div>

Redefinition of structs works fine in Pluto.jl, you can use it as your IDE - notebooks are valid .jl files.

---

<div class="post-metadata">

**Author:** ![thisrod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/thisrod/32/10642_2.png) [@thisrod](https://discourse.julialang.org/u/thisrod)\
**Post date:** [April 9, 2021, 6:28am UTC](https://discourse.julialang.org/t/whats-the-best-practice-in-julia-for-research-often-changed-codes/58892/14 "2021-04-09T06:28:42Z")

</div>

> [@Oscar\_Smith](#):
>
> what happens when I try to call `baz(bar)` ?

In principle, I think that is obvious. Your code should be equivalent to:

```julia
struct Foo1
    x::Float64
end
Foo = Foo1
bar = Foo(1.0)
baz(x::Foo) = Foo(x.x -1.0)

struct Foo2
    y::Int
end
Foo = Foo2
baz(x::Foo) = Foo(x.x -1.0)

```

Although other methods for `Foo2` could work, the method `baz(::Foo2)` will throw an error. No doubt you will eventually get sick of that and redefine it.

It isn’t obvious how the compiler would find all the methods like `baz` that need to be recompiled for `Foo2`. It might be hard to find a compromise that could actually be implemented, but where the semantics were still clean and code did what people expected it to.

---

<div class="post-metadata">

**Author:** ![ndgnuh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ndgnuh/32/23386_2.png) [@ndgnuh](https://discourse.julialang.org/u/ndgnuh)\
**Post date:** [April 9, 2021, 6:48am UTC](https://discourse.julialang.org/t/whats-the-best-practice-in-julia-for-research-often-changed-codes/58892/15 "2021-04-09T06:48:07Z")

</div>

I’m not claiming my workflow is good, but here’s how I do it:

1. Write code in package
2. Have a REPL open near by
3. Write everything I want to try in the test and use `]test`

---

<div class="post-metadata">

**Author:** ![ffevotte](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ffevotte/32/6587_2.png) [@ffevotte](https://discourse.julialang.org/u/ffevotte)\
**Post date:** [April 10, 2021, 8:38am UTC](https://discourse.julialang.org/t/whats-the-best-practice-in-julia-for-research-often-changed-codes/58892/16 "2021-04-10T08:38:41Z")

</div>

> [@nilshg](#):
>
> Another option if you don’t like named tuples is to just call your struct `MyStruct_1` and if you need to change it, do a find replace on your code base to `Mystruct_2` .

> [@jzr](#):
>
> Could a macro do that?

TLDR: I used to have such a macro, but there were quite a few edge cases that I didn’t handle, so I finally gave up using it. (Might still be useful to some people I guess, but I’m not sure if I kept the code somewhere)

First, note that you need to handle not only type definitions, but also type constraints in methods. Taking a simple example with an hypothetic `numbered_struct` macro:

```julia
@versioned_struct v1 struct Foo
  x::Int 
end

bar(::Foo) = 1

```

A simple expansion like this won’t work well with revise:

```julia
struct Foo_v1
  x::Int 
end
Foo=Foo_v1

bar(::Foo) = 1

```

this is because the value of `Foo` is taken into account when `bar` gets compiled, producing a method working on `Foo_v1` instances. But when `Foo_v2` comes into play, no new `bar` method is introduced (unless `Revise` also detects changes to the definition of `bar` itself, in which case it redefines the correct method)

One way to circumvent that could involve abstract types, making the above expand to:

```julia
abstract type Foo end
struct Foo_v1 <: Foo
    x :: Int
end
Foo(args...) = Foo_v1(args...)

bar(::Foo) = 1

```

This way, methods constraining their arguments types to `Foo` actually support any (concrete) version of the (abstract) type. So there is no need to introduce a new method each time a new version of the type is defined.  
This is where I stopped at the time. It does work in some cases, but as I said earlier, I encountered so many edge cases that I eventually stopped using this approach.

(IIRC, one of the main problems was to handle parametric types)

---

<div class="post-metadata">

**Author:** ![ffevotte](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ffevotte/32/6587_2.png) [@ffevotte](https://discourse.julialang.org/u/ffevotte)\
**Post date:** [April 10, 2021, 8:47am UTC](https://discourse.julialang.org/t/whats-the-best-practice-in-julia-for-research-often-changed-codes/58892/17 "2021-04-10T08:47:05Z")

</div>

> [@iHany](#):
>
> I often change my code including code structure and Julia types.  
> When redefining types, Julia REPL needs to be restarted and I should spend quite a long time for precompilation.  
> In my experience, the actual development speed is fast only for almost-completed codes.  
> For some projects on progress, I always restart and precompile the whole code, which is so annoying.

These days, I tend to simply restart my REPL every once in a while when necessary, relying on system images to make this more painless. When I begin working on a large-ish project with large/long-to-load dependencies, I use `PackageCompiler` to build a system image incorporating all dependencies (but not the project I’m developing). That way, the only pre-compilation Julia needs to do at each restart is the precompilation of my own code, which I find to be very fast in most instances.

---

<div class="post-metadata">

**Author:** ![iHany](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ihany/32/18151_2.png) [@iHany](https://discourse.julialang.org/u/iHany)\
**Post date:** [April 10, 2021, 8:55am UTC](https://discourse.julialang.org/t/whats-the-best-practice-in-julia-for-research-often-changed-codes/58892/18 "2021-04-10T08:55:47Z")

</div>

I used to use PackageCompiler for precompilation speed.  
However, it sometimes yields errors and I had to restore images to the default system image.

Haven’t you experienced such errors? I just wonder if it happened only in my case.

---

<div class="post-metadata">

**Author:** ![ffevotte](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ffevotte/32/6587_2.png) [@ffevotte](https://discourse.julialang.org/u/ffevotte)\
**Post date:** [April 10, 2021, 9:10am UTC](https://discourse.julialang.org/t/whats-the-best-practice-in-julia-for-research-often-changed-codes/58892/19 "2021-04-10T09:10:15Z")

</div>

No I haven’t encountered any error lately. Or at least no “structural” error: it does happen sometimes that I run out of memory when I try to build a sysimage while running a memory-intensive computation (which is the politically correct expression for “having too many web browser tabs open”).

Your comment about “restoring the default system image” make me think that we might not have done the exact same thing. To be more explicit: I generate images with a piece of code along the lines of

```julia
using PackageCompiler
create_sysimage(
    [:Plots],
    sysimage_path = joinpath("my_image.so"),
    precompile_execution_file = "precomp.jl",
    # cpu_target = "generic;sandybridge,-xsaveopt,clone_all;haswell,-rdrnd,base(1)"
)

```

that is to say that I never use the `replace_default` keyword argument and therefore never have to use [`restore_default_sysimage`](https://julialang.github.io/PackageCompiler.jl/dev/refs/#PackageCompiler.restore_default_sysimage) if things go wrong.

I then invoke Julia with the `-J my_sysimage.so` command-line switch (either from the command-line or from VScode; this can be configured in the Julia extension settings)

---

<div class="post-metadata">

**Author:** ![Henrique\_Becker](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/henrique_becker/32/15443_2.png) [@Henrique\_Becker](https://discourse.julialang.org/u/Henrique_Becker)\
**Post date:** [April 10, 2021, 12:59pm UTC](https://discourse.julialang.org/t/whats-the-best-practice-in-julia-for-research-often-changed-codes/58892/20 "2021-04-10T12:59:27Z")

</div>

Your idea does not work for parametric types because:

```julia
struct Foo1{T}
    x::T
end
Foo{T} = Foo1{T}
bar = Foo(1.0)
baz(xInt::Foo) = Foo(x.x -1.0)

struct Foo2{T}
    y::T
end
Foo{T} = Foo2{T}
baz(x::Foo) = Foo(x.x -1.0)

```

gives:

```julia
julia> Foo{T} = Foo2{T}
ERROR: invalid redefinition of constant Foo
Stacktrace:
 [1] top-level scope at REPL[6]:1

```

[Next page](https://discourse.julialang.org/t/whats-the-best-practice-in-julia-for-research-often-changed-codes/58892.md?page=2)