Change in local package does not take effect unless Julia restart

New to Julia
1

Suppose I am in a primary environment say, environment/v1.3.
Now I take the following steps to create a package as a local git repo and do some tests:

1) ] generate TestPkg
2) git init/add -A/commit in TestPkg directory
3) ] add TestPkg
4) using TestPkg
5) Try TestPkg.greet()
6) Make some changes on TestPkg.greet()
7) git update on TestPkg
8) ] update TestPkg
9) Try TestPkg.greet() and observe the change takes effect

This is fine and as expected. However if I create a new project environment and repeat the same steps in the local environment, the change on the package source file does not take effect even after git update. It needs Julia restart.

I could not find any clue as to why this is so from Julia doc and Pkg.jl doc.
Am I missing anything?

2

Are you using Revise:

?

2 Likes
3

Sure I am. However, in case that the package is git-controlled, in order for any change to take effect, it needs git commit followed by ] update package (actually this is in line with the statement in Pkg.jl doc). In this case Revise isn’t triggered (but why?). Also, this observation only holds as long as I am in the primary environment. If I change to a local custom environment, I have to restart Julia to see the change each time. Neither Revise nor git commit helps in this case.

On the contrary, any package in dev folder in a primary or custom environment is under Revise’s surveillance if they are in LOAD_PATH (as announced) but strangely for packages in dev folder, git commit does not let any change take effect in the absence of Revise.

4

Can you spell out your steps in detail, including how you’re starting Julia & Revise?

1 Like
5

Thank you for help.

Julia 1.3.0 / Revise v2.3.2

startup.jl is as follows:

using  Distributed

@everywhere push!(LOAD_PATH, "D:/work/julia")
@everywhere push!(LOAD_PATH, "D:/work/julia/test/")
@everywhere push!(LOAD_PATH, "D:/work/julia/test/MyPkg")
@everywhere push!(LOAD_PATH, "D:/work/julia/test/MyPkg/src")

atreplinit() do repl
    try
        @eval using Revise
        @async Revise.wait_steal_repl_backend()
    catch
    end
end

Step 1.

shell> cd test
D:\work\julia\test

Step 2.

(v1.3) pkg> generate MyPkg
Generating project MyPkg:
    MyPkg\Project.toml
    MyPkg/src/MyPkg.jl

Step 3.

shell> git init ./MyPkg
Initialized empty Git repository in D:/work/julia/test/MyPkg/.git/

Step 4.

shell> cd MyPkg
D:\work\julia\test\MyPkg

shell> git add -A

shell> git commit -m"init"
[master (root-commit) a323639] init
 2 files changed, 9 insertions(+)
 create mode 100644 Project.toml
 create mode 100644 src/MyPkg.jl

Step 5.

shell> cd ..
D:\work\julia\test

(v1.3) pkg> add MyPkg
[ Info: resolving package identifier `MyPkg` as a directory at `D:\work\julia\test\MyPkg`.
   Cloning git-repo `MyPkg`
  Updating git-repo `D:/work/julia/test/MyPkg`
  Updating git-repo `D:/work/julia/test/MyPkg`
 Resolving package versions...
  Updating `C:\Users\sijun.cho\.julia\environments\v1.3\Project.toml`
  [48f60d83] + MyPkg v0.1.0 #master (MyPkg)
  Updating `C:\Users\sijun.cho\.julia\environments\v1.3\Manifest.toml`
  [48f60d83] + MyPkg v0.1.0 #master (MyPkg)

Step 6.

julia> using MyPkg
[ Info: Precompiling MyPkg [48f60d83-e34d-427a-8afc-fdeaa2472ade]

julia> MyPkg.greet()
Hello World!

Step 7.

module MyPkg

#greet() = print("Hello World!")
greet() = print("Hello Everyone!")

end # module

Step 8.

julia> using MyPkg

julia> MyPkg.greet() # No change, although paths are pushed to LOAD_PATH hoping Revise will catch
Hello World!

Step 9.

shell> cd test/MyPkg
D:\work\julia\test\MyPkg

shell> git add -A
warning: LF will be replaced by CRLF in src/MyPkg.jl.
The file will have its original line endings in your working directory

shell> git commit -m"upd"
[master 20157e1] upd
 1 file changed, 1 insertion(+), 1 deletion(-)

(v1.3) pkg> update MyPkg
  Updating registry at `C:\Users\sijun.cho\.julia\registries\General`
  Updating git-repo `https://github.com/JuliaRegistries/General.git`
  Updating git-repo `D:/work/julia/test/MyPkg`
 Resolving package versions...
  Updating `C:\Users\sijun.cho\.julia\environments\v1.3\Project.toml`
  [48f60d83] ~ MyPkg v0.1.0 #master (MyPkg)
  Updating `C:\Users\sijun.cho\.julia\environments\v1.3\Manifest.toml`
  [48f60d83] ~ MyPkg v0.1.0 #master (MyPkg)

Step 10.

julia> using MyPkg

julia> MyPkg.greet() # now see the change
Hello Everyone!

If I ] activate myenv and follow the above steps, things are worsen. Even git commit & ] update package no longer works; Julia should be restarted.

6

The short answer is that you should dev MyPkg rather than add MyPkg.

In greater detail: when you say ]add MyPkg,the package manager makes a copy of the files that you created in D:\work\julia\test and stashes them in your .julia/packages directory. Right after the add you’ll probably get something like this:

julia> pathof(MyPkg)
"/home/tim/.julia/packages/MyPkg/o1AMF/src/MyPkg.jl"

This is the path that Revise uses as the definition of this package. Consequently any changes to files in D:\work\julia\test do not get tracked.

Even if you edited the files in .julia/packages/MyPkg/, Revise wouldn’t track them. That’s because add means “use a version-controlled release of a given package.” If you’re making changes, you’re no longer using a version-controlled release. Indeed, in recent Julia versions all the source files in .julia/packages have write permissions disabled. (Your editor might “helpfully” offer to change them to writable, but you should decline such offers of assistance.)

Revise will, however, follow changes from add to dev. If you make a mistake and ]add MyPkg instead, you can fix it by saying ]dev MyPkg and any changes you’ve already made will be incorporated. That’s because Revise will use the files in your .julia/packages/MyPkg directory as a reference against which to compare the changes.

5 Likes
7

Thank you very much for the clear explanation. Now I have come to understand how Revise works.