Not sure if I am using `Revise` correctly

General Usage
1

I am trying to use Revise for a module mymodule defined in the file model.jl.

julia> using Revise
julia> include("model.jl"); using .mymodule

if I change the file model.jl , the changes are not reflected. I imagine that it’s tracking the “module” .mymodule rather than watching for changes in the file. Infact,

julia> Revise.modulefiles(mymodule)
(nothing, nothing)

which, from the docs, suggests

Return the parentfile in which mod was defined, as well as a list of any other files that were included to define mod. If this operation is unsuccessful, (nothing, nothing) is returned.

Is there a way to fix this?

Note that the original poster on Slack cannot see your response here on Discourse. Consider transcribing the appropriate answer back to Slack, or pinging the poster here on Discourse so they can follow this thread.
(Original message :slack:) (More Info)

Mods: Please change title to “Using Revise with pmap, parallel workers, and relative module”

2

The solution here is to use Revise.track.

include("model.jl")
using Revise; Revise.track("model.jl")
using .mymodule

My usecase actually involved using Distributed and multiple workers. For that, the following snippet worked:

# julia> using Distributed; addprocs(4, exeflags="--project=.")
# julia> @everywhere include("model.jl")
# julia> using Revise; Revise.track("model.jl")
# julia> @everywhere using .mymodule
1 Like
3

It’s also worth pointing out that since you’ve gone to the effort to create a module, going one step further to make it available as a package means you don’t need do anything more than

using Revise, MyModel
3 Likes
4

Yes, but how exactly do I do that? I guess I could develop in ./julia/dev and then dev the package in my environment. The problem is that I am using two systems. My local for testing and plotting and then a cluster system for running the main simulations, so it’s just easier to upload my script.jl file and run include. I guess, if I am uploading files, I could upload to the dev folder on the cluster as well.

By the way, @tim.holy could you verify my thought process here?

  • By running using Revise; using .mymodule, Revise is indeed tracking my module. But since there is no file path to this module, Revise won’t actually know when changes are made (i.e. I’ve made changes to a file, not the module per se).

  • By running Revise.track("model.jl"), it’s not tracking both the file model.jl and module mymodule. When I made changes to model.jl, it updates the the included code, but since the included code is simply the module, .mymodule also gets updated.

5

I use PkgTemplates when I first start a new project so the whole directory structure is there and under git version control. I then push my code up to GitHub and use that to deploy to other systems. GitHub gives us, as an academic lab, unlimited free private repositories.

6

GitHub has unlimited private repositories for unpaid users too (Pricing · Plans for every developer · GitHub).

My experience is also that packages win almost in all cases. With packages, you get many benefits almost for free, such as

  • setting up tests and documentation
  • using your code in other places
  • dependency management (via Project.toml)

Also, you don’t have to put much in ~/.julia/dev. Instead, use

julia> ]

pkg> activate .

(MyPackage)> # Type a backspace here to go back to julia>

julia> using Revise; using MyPackage
2 Likes
7

Well, isn’t that still slightly more complicated?

I develop a package on my local, in the ./julia/dev folder. This is great, as locally all I have to do is using Revise, MyModule and just keep working. I can even occasionally push to github if the folder is git initialized.

But my workflow is slightly different. The problem for me is that, often I make a change, I test it out locally, but then need to immediately upload to a cluster, and test out my changes at a much larger scale (think calibrating a large agent-based model where I need to run a few hundred realizations to get meaningful results).

So with my current solution, all I have to do is work locally, save the file, and upload to cluster (one command in vscode extension). And because I am tracking the file model.jl using Revise, my code is automatically updated (not only on the main process, but also on the worker processes as well). I find this workflow very efficient.

Using it the “package” way, I have to push to github even after the smallest change, then up on the cluster, restart Julia, and using MyModule again. Revise dosn’t end up being useful for me.

8

No, you can have a branch, then have Revise open on both the cluster and your local computer.

Push and pull from the branch only, and Revise will track both just fine.

EDIT: Don’t use up, use the git repo directly to manage updates.

9

Okay, thanks. I will work on a MWE and update this thread and even submit a documentation PR.

3 Likes
10

It seems to me that “packages” that Revise can track don’t have to have the full blown package structure that Pkg knows how to work with. I.e. a “package” consisting of a file named “MyModel.jl” (with possibly included files) in a directory in the LOAD_PATH and containing a module MyModel can be tracked with Revise via using Revise, MyModel.

1 Like
11

@affans, there are several viable solutions here, so you should just pick the one that works for you. Your first post didn’t spell out all your constraints, so I and others suggested alternatives to the solution you found, but in the end you know what works for you best.

1 Like