Separate "Dev" packages in Project.toml

General Usage
1

When creating a package, I often use dependencies that are purely for development and have nothing to do with my package’s functionality (e.g. Revise.jl, Debugger.jl, Infiltrator.jl. If I ever wanted to share my package, these unnecessary packages would be shared too unless I manually deleted them from the Project.toml.

Pipenv (for Python) makes a distinction between “dev” dependencies only needed for development ([dev-packages] in the Pipfile), and strictly necessary dependencies needed for running the project code ([packages]). A user can run pipenv install to install only the packages needed for running the code and ignore the “dev” packages. If they want to install the dev packages too, they can use pipenv install --dev.

I couldn’t find any similar functionality in the documentation for Pkg.jl. Can Pkg also distinguish between development and strictly necessary packages when installing a project environment?

3 Likes
2

This is a good question.

My current approach is to have Infiltrator in my 1.6 project.toml, then inside my module code use Main.@infiltrate.

1 Like
3

Can all of that stuff go in test dependencies? Having the separate project and manifest for those seems to make sense and presumably you are doing development by editing/creating tests anyways?

1 Like
4

Environments in Julia are nested, when working on a project you usually have access to both - package’s dependencies and dependencies in the enclosing environment (e.g. @v1.6):

(YourPackage) pkg> st
# prints dependencies of YourPackage

(YourPackage) pkg> activate.  # switch to the default environment
  Activating environment at `~/.julia/environments/v1.6/Project.toml`

(@v1.6) pkg> add Revise
# adding Revise to the enclosing environment

(@v1.6) pkg> activate .   # switch back to the environment in the current directory 
  Activating environment at `...`

julia> using Revise
...
8 Likes
5

The way most people deal with this is to have all those dev packages, as you call them, installed in your Main environment ((@1.6), for example) and then create a new environment for your package ((@MyPkg)) . With this setup you have access to Revise and Debugger, and any other package you need to work on your package, but they are not part of the Project.toml file of your package.

1 Like
6

So, technically, there are two separate @MyPkg environments, one in environments folder and the other in dev/MyPkg folder? Is there a workflow description for this thing?

7

It’s documented in the Pkg.jl docs introduction here. Basically, when you start the REPL with julia --project, you have access to your 1.6 environment as well as your MyPackage project.toml.

This issue is about how stuff inside MyPackage can’t see the 1.6 project.toml

8

But no. There are not two MyPkg environments. People above this post have been referring to the Main environment (@v1.6 for example) and your package’s environment @MyPkg. The packages that are installed in the main environment are accessible from all julia sessions no matter what other environment you have active.

2 Likes
9

2 posts were split to a new topic: Simultaneous development of private packages?

11

You can build your own custom stack pretty easily (as briefly mentioned above). For example, if you keep dev tools installed in devtools/Project.toml as follows:

$ tree .
.
|-- devtools
|   |-- Manifest.toml
|   `-- Project.toml    # Debugger, BenchmarkTools, ...
|-- Manifest.toml
|-- Project.toml        # Regular dependencies
`-- src
    `-- MyProject.jl

Then you can set up the Julia load path to first look in Project.toml and then devtools/Project.toml (and then in stdlib):

$ export JULIA_LOAD_PATH="${PWD}:${PWD}/devtools:@stdlib"

To handle this automatically I recommend using direnv. I wrote a blog post about direnv and Julia which has some more examples of this (and other nice things you can do with direnv together with Julia).

4 Likes