Where to store .jl files

[alan]

Julians seem to keep their .jl files under ~/.julia/dev. Why is this? Is this location used somehow by “import”, “using”, or “include”?

In C, I’m used to keeping source files under ~/src, then copying executables into ~/bin and libraries into ~/lib. Then I can use the executables and libraries from anywhere. In Julia, I have not yet discovered a similar system.

[kellertuer]

I have my source files either in a package, that is the package‘s src/ folder or a specific repository/folder with example scripts (usually then either Pluto or Quarto notebooks).
And I usually start (at least a local) package as soon as I want to share common code between notebooks. But the package would still be in my Julia Code default folder (Repositories/Julia).

@alan

Julians seem to keep their .jl files under ~/.julia/dev.

So could you maybe state where your assumption is from? I would feel strange storing generic stuff I use in a hidden folder.

[lostella]

@alan

Julians seem to keep their .jl files under ~/.julia/dev

That’s where Pkg, the package manager, by default stores registered packages you dev (install for development).

This section of the documentation explains how Julia code can be loaded, implying how one could go about organizing it. One generally wants to use environments, which are covered here. See also the section in Pkg’s documentation about managing environments.

[goerz]

] dev name_of_package always seemed like a bad feature to me.

If I want to dev-install something, I’ll manually clone its repository to ~/Documents/Programming and the do Pkg.develop(path="~/Documents/Programming/name_of_package"). The ~/.julia folder is best kept transient so that you can delete it without losing any important data.

I’m not sure if “keeping .jl files under ~/.julia/dev” is all that common in the wider Julia community, but I’d certainly advise against it.

[kellertuer]

When you start Julia in the base folder of your project ] dev . is a nice shortcut to take a (locally checked out) package in development mode.

[goerz]

Sure, but that doesn’t put anything into ~/.julia/dev, right? At least I think it’s equivalent to the more verbose Pkg.develop(path=".") which makes the current environment use the package implementation in the current working directory, and which I definitely use myself.

I’m not opposed to the shortcut ] dev path, but ] dev name for a package name that then gets cloned into ~/.julia/dev/ is not so great. Nor that ] dev something is ambiguous whether something is a path or a name. That’s why, personally, I always use the explicit Pkg.develop, but it’s a separate gripe from whether or not to store code in ~/.julia/dev.

[goerz]

Actually, there is one use case where dev-installing into ~/.julia/dev can be very useful: debugging. If there’s something weird going on in some random package that I’m not the developer of, it’s an easy way to make sure the weird behavior is still present in the master branch, and to add some debug-print statements. However, this is very temporary: I’ll delete ~/.julia/dev and the Manifest.toml of my current environment when I’m done with debugging. And, if I’m interacting with that package over a longer period of time, it’s going to get cloned to a more permanent place in ~/Documents/Programming.

[mkitti]

In short, you can put your Julia source files wherever you want. ~/.julia/dev just happens to be a common default.

If you like to access them from ~/src, you could always do ln -s ~/.julia/dev ~/src/juliapkgs on a *nix system.

As others have explained, Pkg.develop and other tools such as PkgTemplates.jl default to populating ~/.julia/dev. You can change this default by setting the environment variable JULIA_PKG_DEVDIR.

You mentioned executables in ~/bin and libraries in ~/lib. Julia workflows usually do not involve producing a lot of files that would be appropriate to put there. One exception may be using PakageCompiler.jl to produce apps, system images, or libraries.

As of Julia 1.9, Julia does produce native shared libraries (.so files) in ~/.julia/compiled/v1.9 but those are not really meant to be loaded outside of Julia.

Perhaps the most common workflow that may involve placing files in ~/bin are executable scripts. I’ll show a minimal example of an “executable” project package.

$ tree -p src/HelloWorld.jl/
[drwxrwxr-x]  src/HelloWorld.jl/
├── [drwxrwxr-x]  bin
│   └── [-rwxrwxr-x]  helloworld.jl
├── [-rw-rw-r--]  Project.toml
└── [drwxrwxr-x]  src
    └── [-rw-rw-r--]  HelloWorld.jl

2 directories, 3 files

$ cat src/HelloWorld.jl/src/HelloWorld.jl 
module HelloWorld
    __init__() = println("Hello World")
end

$ cat src/HelloWorld.jl/bin/helloworld.jl 
#!/bin/env julia
using Pkg
if length(ARGS) > 0 && ARGS[1] == "-q"
    io = devnull
else
    io = stdout
end
Pkg.activate(
    @__FILE__() |>
    x->(islink(x) ? readlink(x) : x) |>
    dirname |>
    dirname
    ; io
)

using HelloWorld

From Julia, one may load this package in the following way.

$ julia --project=src/HelloWorld.jl -e "using HelloWorld"
Hello World

If we just want a generic executable script, then we can use the helloworld.jl file.

$ src/HelloWorld.jl/bin/helloworld.jl 
  Activating project at `~/src/HelloWorld.jl`
Hello World

Now you may want this always on your shell’s path. Assuming that ~/bin is on your shell’s path, you could then do the following.

$ ln -s ~/src/HelloWorld.jl/bin/helloworld.jl ~/bin

$ helloworld.jl 
  Activating project at `~/src/HelloWorld.jl`
Hello World

Additionally, I can also load this package into another Julia environment. Below I use Pkg.develop to incorporate it into a temporary Julia environment.

$ julia -e 'using Pkg; Pkg.activate(; temp=true); Pkg.develop(path="$(ENV["HOME"])/src/HelloWorld.jl"); using HelloWorld'
  Activating new project at `/tmp/jl_XxWuFq`
   Resolving package versions...
    Updating `/tmp/jl_XxWuFq/Project.toml`
  [ba1a46c8] + HelloWorld v0.0.0 `~/src/HelloWorld.jl`
    Updating `/tmp/jl_XxWuFq/Manifest.toml`
  [ba1a46c8] + HelloWorld v0.0.0 `~/src/HelloWorld.jl`
Hello World

Part of the reason I have the code arranged as a package is the because is the basic cacheable compilation unit in Julia.

$ ls ~/.julia/compiled/v1.9/HelloWorld
QoJsw_rFRaA.ji  QoJsw_rFRaA.so

I hope that helps.

[alan]

Thanks to everyone who replied. These comments were extremely helpful.