What's the best practice in Julia for research (often changed codes)?

[iHany]

Yes, actually I’d tried to use PackageCompiler very early, and I might have been confused with how to use it.
I’ll try your suggestion (without replace_default=true).
Thanks

EDIT: do you execute the precompilation code in your custom environment (pacakge)? Or, in default env?

[ffevotte]

@iHany

do you execute the precompilation code in your custom environment (pacakge)? Or, in default env?

I usually have a dedicated build sub-directory defining its own environment (much like how things are traditionally set up for using Documenter in the docs subdirectory)

MyProject
├── build
│   │   # PackageCompiler-specific environment
│   ├── Project.toml  # add PackageCompiler
│   ├── Manifest.toml # dev ..
│   │
│   ├── make.jl       # script calling `create_sysimage`
│   └── precomp.jl
│
│   # Normal project environment
├── Project.toml
├── Manifest.toml
│
└── src
    └── MyProject.jl

I’ve actually been thinking of adding a plugin to set this up in PkgTemplates; now might be a good time to do it

[jzr]

@ffevotte

But when Foo_v2 comes into play, no new bar method is introduced (unless Revise also detects changes to the definition of bar itself, in which case it redefines the correct method)

I thought redefining the method would be good and Revise should do that, no?

[ffevotte]

IIUC, when Revise sees something like

bar(::Foo) = 1

it does not keep track of the fact that the method definition depends on what Foo happens to be bound to at the time when the method definition was evaluated. This makes sense because Julia types are normally constant. But if Foo happens to be a variable and it gets re-bound to a new value (a different version of the type), then Revise currently has no way of knowing that it should re-evaluate the definition of this particular method of bar.

However, Revise will re-evaluate the definition of bar(::Foo) if its source code changes.

You can try and see what happens when you put some code like this in a package

module MyPackage
   struct Foo_v1 end
   Foo = Foo_v1

   bar(::Foo) = 1
end

and then in a REPL:

julia> using Revise
julia> using MyPackage
[ Info: Precompiling MyPackage [ae553910-327c-57b0-bdb1-851592e5e7e1]

julia> MyPackage.bar(MyPackage.Foo())
1

# At this point, edit `MyPackage.jl` to replace `Foo_v1` with `Foo_v2`
# Revise will be happy with such changes, but...

julia> MyPackage.bar(MyPackage.Foo())
ERROR: MethodError: no method matching bar(::MyPackage.Foo_v2)
Closest candidates are:
  bar(::MyPackage.Foo_v1) at /tmp/MyPackage.jl/src/MyPackage.jl:5
Stacktrace:
 [1] top-level scope
   @ REPL[3]:1

Revise did account for the change in the Foo constructor (the source code of its definition was updated to point to Foo_v2), but not bar (its source code was left untouched).

[jzr]

@ffevotte

Revise currently has no way of knowing that it should re-evaluate the definition of this particular method of bar .

That seems like it would be a useful feature for Revise. Is there a downside?

[ffevotte]

@jzr

That seems like it would be a useful feature for Revise.

I think everybody would agree with you!

@jzr

Is there a downside?

We’ve more or less reached the limits of my understanding of how these things work, but here are two discussions that might be of interest to you (and lead to other places where such matters are discussed)

redefining struct

Proposal: Partition bindings by world age

[thisrod]

@Henrique_Becker

Your idea does not work for parametric types because:

...
Foo{T} = Foo1{T}
...

That doesn’t work, but it wasn’t my idea. As far as I can see, the Foo = Foo1 version works fine with a parametric type. The whole point is for Foo to be a variable, so that reevaluating the method definitions adds extra methods instead of changing the existing methods.