This is a bit of a soft/vague question, but I am struggling to find a good, informal, workflow for my budding Julia development. I’m probably just tripping over my own preconceptions, please point me in the right direction.
Let’s say I want to use Revise.jl. It seems I have to put my code in a Module to make it work. The Julia documentations Workflow Tips suggest using include, but that does no seems to sync with Revise. To be able to load the module, it seems easiest to put it in a Package. The package documentation talks about projects, but I’m not quite sure what the distinctions are. To have local packages, it seems I have to create git repositories for them.
Walking down this road, it seems to me that Module = Package = (magic directory structure) ≈ Project = git repo. But that can’t be right. And surely Julia development is not cemented on git?
Are there any blogs or tutorials on how to manage a small julia project? Currently just a few own local modules, lots of editing and restructuring going on. How to organically grow a project from one file to many modules? How should I structure my directories/modules/packages, the whole lot!
It could be that I misunderstood Revise’s docs.
This is how I use it (using the REPL workflow example): Tmp.jl:
module Tmp
export say_hello
say_hello() = println("Hello!")
# your other definitions here
end
then in Tst.jl:
using Revise
includet("Tmp.jl")
using .Tmp
say_hello()
# say_hello()
# your other test code here
It’s working for me. But it could be that after using Revise the use of includet() is redundant, as the Tmp module is will be tracked by Revise after the using statement.
Also sorry for not answering directly your questions, I misinterpreted something.
Unfortunately Revise.jl doesn’t handle changes in type definitions.
I would like to know if there is a better workflow to handle these kind of changes.
Personally I make modules only when it makes sense to. That is when I have a clearly defined and atomic set of features that I want to reuse often. Otherwise, I make scripts that use modules and often load (via include) a “library script” (a file that contains mostly functions).
Basically all code starts in a script at the global scope, if it’s useful it goes into a “library script”, and if that is really useful it ends up in a package.
For reloading type definitions I often just search and replace the type name (adding a version number to it).
Note that you can also reload a module (including type definitions) just using include or with plain code if you don’t use exports for testing:
module Foo
bar(x) = 2x
struct MyStruct
x
end
end
Foo.bar(2)
Foo.MyStruct(1)
I usually start my modules like that and then move them to a package. A package is a special directory structure (Foo/src/Foo.jl) containing a module Foo and added to your environment via the package manager. They don’t require git.
Thank you for your answer! I also found your blog and blogpost: My Julia workflow, and I realised that I have much to learn from you.
One follow-up question: Why do I have to pkg> activate every time I want to run anything from the project? Isn’t all the waltzing with Manifest.toml and Project.toml enough? Or putting it in a different way, what variable does the activate update and how does it enable julia to find my code? I thought the activate was only for Pkg and that code loading relied on the .toml files.
For example, in a newly created directory TestProj:
This works and gives me a false sense of having set up a working environment. But if I exit and re-enter julia:
> using TestMod
ERROR: ArgumentError: Package TestMod not found in current path:
- Run `import Pkg; Pkg.add("TestMod")` to install the TestMod package.
I have to ] activate . again merely for using to work, even if no Pkg managing actions are to take place.
I apologise for the long post, I’m just trying to expose my thinking so that you people can point out the flaws in it.
It changes the active environment (again, please do read the Pkg docs, they clarify a lot).
To give an example, suppose I am working on code for a research paper. It will not be used by anything else, but uses quite a few packages. I also want to commit the Manifest.toml, so that I can always rely on a consistent and tested environment, and upgrade on demand (eg I run it on a remote server so I want it to be rock-solid).
So I generate a package skeleton in ~/research/CodeForPaper. This is not in my global environment.
Whenever I want to work on it, I pkg> activate ~/research/CodeForPaper. I occasionally pkg> up then, which updates this environment. After testing I commit to the repo. Otherwise it does not interfere with anything, and is practically “invisible” unless I activate it.
Thank you for your patiens. I have read the Pkg docs but probably misunderstood them. The sentence
You may have noticed the (v1.1) in the REPL prompt. This lets us know v1.1 is the active environment. The active environment is the environment that will be modified by Pkg commands such as add, rm and update.
gives me the impression that the “active environment is the environment that will be modified by Pkg commands such as add, rm and update”. Apparently the active environment is also the environment that determines what packages are loaded. I thought the toml-files had something to do with that. Project environments:
A project environment is determined by a directory containing a project file called Project.toml, and optionally a manifest file called Manifest.toml.
Now feel that these files are just dead in the water until I activate the project. Is that correct?
More or less (except for resolving the dependency graph if necessary, if I understand correctly), except for stacked ones (but I would not worry about that at the moment).