Package development tools - in environment but not dependency

General Usage Performance
,
1

Hi,

I am getting a package EspyInsideFunction.jl fairly formalized (testing, doc, registration…). But I still wonder about one thing: as part of development work I use Documenter.jl, Test.jl and ProfileView.jl, so they must be in the package’s environment. But I do not want to force these tools to be installed as dependencies on the computer of someone using my package, so they must not be in the package’s environment.

Please correct my statements were needed (there has to be one or several correction, because things do not work as I describe them!)

  1. When developing (testing, documenting) a package, I activate that package’s environment. I do not use a separate “develop_EspyInsideFunction” environment.
  2. In the Project.toml file, I have
name = "EspyInsideFunction"
uuid = "d6b55740-5cb4-11ec-1cb9-93db9984c461"
authors = ["Philippe Maincon <philippe.maincon@sintef.no>"]
version = "0.2.0"

[deps]
Printf = "de0858da-6303-5e67-8744-51eddeeeb8d7"

[compat]
julia = "1.6"

[extras]
Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"
Documenter = "e30172f5-a6a5-5a46-863b-614d45cd2de4"

[targets]
test = ["Test"]

[extras] is for development tools that are not dependencies of my package.

  1. To get Test and Documenter in [extras] I first ] add these packages, then manualy move the references from [deps to [extras] in Project.toml.

As said, that’s in my dreams. With Project.toml as above, launching the test suite (including doctests) crashes:

ERROR: ArgumentError: Package Documenter not found in current path:
- Run `import Pkg; Pkg.add("Documenter")` to install the Documenter package.

What am I doing wrong?

:grinning:

2

As soon as you move the test dependencies from [deps] to [extras], they are no longer available in the package environment EspyInsideFunction. I have found two solutions for that:

  • Use TestEnv.jl to activate your test environment and develop in it.
  • Put some common test dependencies in your base environment @v1.7. Of course you still have to keep them in [extras], but if they are packages that you commonly use then it may be worth it. I do this for lightweight packages related to documentation, profiling, benchmarking, etc.
3

Without having tried it out, TestEnv.jl allows to make the [extras] available for testing only, using e.g.

using TestEnv, ReTest
TestEnv.activate("Example") do
    retest()
end

Looks perfect, I’ll try that. Oh and ReTest.jl is worth trying.

About your second suggestion, for the sake of better understanding environments: you propose to use a separate enviroment @v1.7 in which Test.jl etc. are installed. To test, one would activate this @v1.7 environment and runtest. Two questions:

  1. I would need to ] dev EspyInsideFunction in that environment, right?
  2. In @v1.7, Test.jl would have to be in [deps], not [extras] - otherwise these packages will not be available (unless using TestEnv. But why do you say " ou still have to keep them in [extras]"?

:slight_smile:

4

About the second suggestion, @v1.7 is actually the name of your default Julia environment, the one that pops up when you run julia in a terminal. This base environment is always stacked underneath any activated environment, so that packages installed there are available everywhere

5

For instance, here’s what I have in mine:

(@v1.7) pkg> st
      Status `~/.julia/environments/v1.7/Project.toml`
  [6e4b80f9] BenchmarkTools v1.3.1
  [e30172f5] Documenter v0.27.18
  [98e50ef6] JuliaFormatter v1.0.2
  [5fb14364] OhMyREPL v0.5.12
  [65465c31] PackageCompatUI v1.0.2
  [14b8a8f1] PkgTemplates v0.7.28
  [c3e4b0f8] Pluto v0.19.5
  [295af30f] Revise v3.3.3
  [1e6cf692] TestEnv v1.7.3
6

It’s the stacking idea that I had missed: so having packed my default envronment with development tools, I activate the environment of EspyInsideFunction, and run all my tests.

An implication is that I must be very careful not to have any packages in my base environment, that could be a dependency of my package: I might forget to add the dependency to my package environment, yet have my tests succeed. So for example, if I want to have an environment packed with plotting and algebra goodies (think of PyLab), for REPLing and scripting, I will make very sure it’s not the deffault @v1.7.

I think that’s my preferred solution. Thanks a bunch!

1 Like
7

Another reason you don’t want your base environment too packed is that it can cause compatibility problems during stacking. I’ve encountered this a few times, which is why I keep it light nowadays

1 Like