How should I use Revise.jl (and other developer tools) to develop a package?

When I create a new package, I often want BenchmarkTools, Revise, and other dependencies that are accessible within my package’s environment, but won’t be added as dependencies for users. I’ve tried playing around with [extras], but I have not been able to replicate this usage.

How are folks using Revise when developing a package? Are you adding Revise as a dependency, then removing it before you commit?

Did you try putting it in the global environment?

2 Likes

I work in a global (or my own separate) environment when developing a package and have things like Revise.jlin said environment. In that environment I also have the package I develop in development mode and then everything works smoothly, without the need of having Revise or any other tools you use in the dependencies of the package you develop.

My habit is to put Revise.jl and friends in the global environment @v1.9. Then I activate the project environment MyPackage/Project.toml (in the folder where I cloned the package). I rely on environment stacks which automatically make packages available from @v1.9 in the project environment.

@kellertuer could you explain what additional benefits your option brings? I’m curious to learn if there is a better path

2 Likes

It seems our difference might be that you activate the projects environment while I put it in development mode?

My option has the (original) advantage that just using MyPackage in the environment where MyPackage is in development mode loads the source from said folder (instead of somewhere from .julia/.... Maybe one advantage is that this way I see the package “from the outside”, i.e. if you think of having a package B that the package uses internally but my global environment does not, then I know whether this works for the user later?
I am not 100% sure, this is mainly how I learned this stuff.

Activating the project environment of the package clone actually does the same thing vis-à-vis loading the source. TBH pkg> dev has always been mysterious to me, so I remain wary of incantations beyond my mortal understanding

Then they might be more similar then I thought.

I think a main difference then is, that

  • In development mode I am in a different environment than the internal project one
  • a package stays in development mode, that is restarting Julia and having 2-3 packages in development mode, keeps them in that mode.

The main thing that dev mode does is really just loading the source from the folder you cloned and not from the copy in the .Julia/ folder.

I am not sure whether 2-3 packages in dev mode is that easy with just activating environments?
For me that is a use case when I for example change something in ManifoldsBase.jl, Manifolds.jl and want to adapt Manopt.jl as well – or at least check what effect these changes have. Then all 3 have changes that should be taken into account, when using them.

1 Like

OK the multi-package development workflow seems to be a use case where your method scales better. With mine I would need to resort to temporary dev-ing ManifoldsBase into the Manopt environment

Sure, if you have a main one that is possible.

In my setup, this is a bit complicated, since Manifold is not a dependency of Manopt and vice versa, so I would have to change dependencies for a while for that.
Then I would not be sure how that interacts with the fact that Manifolds is a weak dependency of Manopt.

So – for single packages I think both variants are the same – besides that dev mode stays after Julia restarts and you have to reactivate your env. For multiple packages, this might be different.

The dev workflow is indeed needed when you have multiple dependent packages you want to develop or if you want to run tests on some package while working on a dependency.

For the single package scenario I find it easiest to

  • Change directory to wherever I have the package (which I want to do anyway for git operations).
  • Start with julia --project to activate the package. Should I need to restart Julia it’s only an arrow up + enter to get back to the activated project.
1 Like

@gdalle and @kellertuer, in your experience, which approach is best for having a nice test writing experience?

My workflow derives mainly from setting things up using PkgTemplates. However, one side effect of this is that VS Code with Julia support does not provide intellisense/auto-completion for the package I am writing the tests for.

Do you have a suggestion for a better experience when writing tests? (if you think I am not on topic, I can create a new one).

I think I set my packages up without the PkgTemplates (or it is just really long ago),
but in my mode with the development mode, sure you have autocomplete for all content of the package you develop, also when writing the tests.

1 Like

Thanks to your comment I discovered that my IntelliSense doesn’t work the way I expected, so I’m afraid I’ll have to understand that before I answer ^^

2 Likes

I would also want to get to the level of proficiency and short-term memory where the Intellisense breaks and I don’t even realize :slight_smile:

I think it’s worse than that, I think I’ve never used IntelliSense because it’s always been broken for me ^^

1 Like

Maybe I should try to drop the intellisense altogether for a while - it would be like altitude training for endurance, but in Julia context.

1 Like

I concur with your diagnosis @algunion, see my investigation here:

I think if @kellertuer is correct I will switch to pkg> dev-ing the package I develop in a shared environment. However I discovered that this has issues too:

1 Like

For the development mode, as far as I remember, I do not get autocomplete just because a file is in the project folder, but as soon as it is (directly or transitively) included in the main MyProject.jl file, but I have not yet looked into this when using extensions.