Revise and include order in package under development

General Usage
,
1

Hello,

After I refactored my own set of code files into a package (with generate command), and using Revise prior to everything (thanks to the VS Code option), development got better, but I noticed some changes in my submodules were not tracked, compared to other changes in other same level submodules.

After I removed almost all my main code in order to create a minimal working example, I found the source of the problem…

Module tree:

- test file.jl
- Project.toml…
- src
  - MyModule.jl
  - submodule1.jl (intermediate level functions)
  - subsubmodule1.jl (low level functions)
  - subsubmodule2.jl (low level functions)

test file.jl:

using MyModule

test_all();

MyModule.jl:

module MyModule

include("subsubmodule1.jl")
include("submodule1.jl")  # Includes after this line are not tracked anymore, so order seems important in this setting.
include("subsubmodule2.jl")

using .SubSubModule1
using .SubSubModule2

export test_all

function test_all()
    test_0 = "b"
    @show test_0
    test1()
    test2()
end

end

submodule1.jl:

module SubModule

include("subsubmodule1.jl")
include("subsubmodule2.jl")

end

subsubmodule1.jl:

module SubSubModule1

export test1

function test1()
    test_1 = "b"
    @show test_1
end

end

subsubmodule2.jl:

module SubSubModule2

export test2

function test2()
    test_2 = "b"
    @show test_2
end

end

When I execute test file.jl, as expected, the result is:

julia> using MyModule

julia> test_all();
test_0 = "b"
test_1 = "b"
test_2 = "b"

After I changed all "b" by "a", this is the result:

julia> test_all();
test_0 = "a"
test_1 = "a"
test_2 = "b"

After several tests, it seems all includes occurring after the include of the submodule1 (including again the 2 subsubmodules) are not tracked by Revise, so each change in these files needs a restart of Julia…

In this example, it seems easy to solve, at least by changing order of includes, but in my real files, these includes are mainly nested in intermediate level modules, so high level modules will call low level modules several times (and the interfering includes are not always in the same file)…

To give a more real code tree:

- test file.jl
- Project.toml…
- src
  - MyModule.jl
  - submodule1.jl (includes subsubmodule1.jl, subsubmodule2.jl, subsubsubmodule1.jl and subsubsubmodule2.jl)
  - submodule2.jl (includes subsubsubmodule1.jl, subsubsubmodule2.jl and subsubsubmodule3.jl)
  - submodule3.jl (includes subsubsubmodule1.jl, subsubsubmodule2.jl and subsubsubmodule3.jl)
  - subsubmodule1.jl (includes subsubsubmodule1.jl and subsubsubmodule2.jl)
  - subsubmodule2.jl (includes subsubsubmodule1.jl and subsubsubmodule2.jl)
  - subsubsubmodule1.jl
  - subsubsubmodule2.jl
  - subsubsubmodule3.jl

And in this real case, changes in subsubsubmodules are not tracked, but if I comment the includes of submodules 1 and 2 (removing the multiple includes), they are tracked again after the restart of Julia.

Did I miss something in the guides for Revise and Pkg for development, do I make it the wrong way with inner imports/includes?

More generally, what should be improved in this code tree?

Thank you for your answers, and great work!

Sincerely.

2

I think that including a single file multiple times will in general leads to trouble.

2 Likes
3

Thank you for your short answer, it made me find this topic.

I removed all my includes and put all of them once in the MyModule.jl file, in reverse order (from lowest level to highest), then I imported the modules with double leading periods in other files, and it seems to work!

Strangely, the number of leading periods does not seem to change anything (at least 2, but it can be quite random, which surprised me…).

MyModule.jl:

include("subsubsubmodule1.jl")
include("subsubsubmodule2.jl")
include("subsubsubmodule3.jl")
include("subsubmodule1.jl")
include("subsubmodule2.jl")
include("submodule1.jl")
include("submodule2.jl")
include("submodule3.jl")

using .SubModule1
…

subsubsubmodule2.jl:

using ..SubSubSubModule1
…

I am still not sure if it is the best (cleanest) way to do, but it seems not bad. Maybe I will later find some drawbacks, but I will keep this way for now.