How to use VSCode and REPL to write and test a package?

Tooling VS Code
,
1

I have gone through a multitude of tutorials now but still can’t seem to wrap my head around the basics of how to write and test a package as I go. I don’t understand the proper way to add my package and its dependencies to the REPL namespace or the proper way to juggle between the Main and MyPackage modules. I keep getting errors that variables, functions, or packages are undefined. I just want the simplest, foolproof workflow.

  1. Open MyPackage folder in VSCode made with PkgTemplates.jl.
  2. Start REPL
  3. Activate MyPackage environment (usually done automatically).
    I should always stay in MyPackage environment when developing MyPackage right?
  4. I don’t need to add or using Revise because it is built in to the VSCode extension right?
  5. Should I using MyPackage, using MyDependency, using .MyPackage, using .MyDependency, or some combination?
    I would assume that loading MyPackage would also load its dependencies, but that doesn’t always seem to be true. I have also seen some tutorials not using MyPackage at all and just send code to the REPL.
  6. How do I ensure my REPL is testing with the package versions in my Project.toml and using the MyPackage version from my .julia/dev folder?
  7. Write MyPackage.jl: module MyPackage; using MyDependency; include("myfile.jl"); end
    Should I use include or includet here?
  8. Start writing code in myfile.jl
  9. How should I debug?
    If I “Run and Debug” MyPackage.jl, then breakpoints in myfile.jl are skipped and I just see “Done!”. If I “Run and Debug” myfile.jl, then MyPackage and MyDependency are not loaded and I error out.
  10. How should I run file code and tinker in REPL?
    I like to switch between writing code in file (executing with Ctrl+Enter) and writing code directly in the REPL, but that sometimes switches the current module between Main and MyPackage so things break. Do I have to stick with one or the other? Which module should my REPL be using?
  11. How should I test sub-functions?
    I can test a function in the REPL with using MyPackage; f(x). However, if that function contains a sub-function f_sub(y), the REPL doesn’t have access to its name so I can make sure it works.
  12. How should I write file paths so they always point to my test data stored in MyPackage/test/testdata/?
    I have to change the file path if I run code from the test folder or if I move the package.
11 Likes
2

This is what I do:

1, 2, 3) First time: go to the package directory and run code . (I find that easier than navigating with mouse clicks). Next time when one opens VSCode just open the project from the list of previous projects that I am working at.

  1. No need for using Revise, but I have it in my julia startup file, so maybe I’m mistaken there.

  2. using MyPackage only. If you need something else, that means that something is not quite correct about your dependencies. Note that you need to add the dependencies to the project with add Dependency.

  3. ] status MyPackage tells you which is the version.

  4. Definitely do NOT use the t. The file will be tracked because it is called from within the using....

8 - 10) not sure

  1. Probably yes. The function f that is exported will know about f_sub, because they share the scope of the package. But if you want to test f_sub only, you need to use MyPackage.f_sub, or import MyPackage: f_sub and then use f_sub.

  2. Use relative paths, i. e. data = "./testdata/mydata.dat inside the scripts of runtests.jl.

3

I never do that :slight_smile: I normally activate some global env like ]activate @mypackagedev, then I dev MyPackage into that with dev . The benefit of that is that I can now add dependencies to this global mypackagedev project that I want to be available at the REPL without having to add them as a dependency of MyPackage.

We do not ship Revise.jl out of the box, we just try to load it when it is available (this might change in the future, I got pretty far at some point with shipping it out of the box). I typically add Revise.jl to may main default v1.7 project. That project is always available, even if you activate another project because it is always in the LOAD_PATH. The VS Code extension will then automatically try to load it into the REPL (there is a config setting to control that).

I normally have some other Julia file somewhere else where I can write test code that I then execute in the REPL, and I don’t commit that file. I can then send code from that file to the REPL etc.

Make sure you use @__DIR__ with relative paths. So it might look like path = joinpath(@__DIR__, "../test/testdata")

5 Likes
4

https://github.com/JuliaTesting/TestEnv.jl is the only reproducible way I have found. Not sure who originally created it, but I think it was another beautiful package from @oxinabox

  1. Create your normal package Project.toml file, no manifest checked into git
  2. Create a /test/Project.toml file, again best not to check a manifest in and to use the compat as required, but less of an issue there,
  3. Do a ] add TestEnv to the global environment. Hopefully vscode even add/load testenv automatically at some point in the future.
  4. Otherwise, keep your global environment clean (e.g. Revise, PkgBenchmark, BenchmarkTools, TestEnv but not much else) usually works best. I keep the package I am working on out of it, but probably other ways work

Just to be clear: at no point do you need to add your package to theglobal environment or manually make any temp project file, and you definetely don’t want to put test-only dependencies in the global environment,

Then to use this:

  1. Start vscode in your packages folder so it starts activated.
  2. To run the unit tests fully, you could just do the classic ] test and it uses the test/Project.toml
  3. To work interactively, given that your package is activated just go using TestEnv; TestEnv.activate() when you start up the REPL,

This creates, activates, and instantiates a temporary manifest that emulates what ] test would do. Fully reproducible.

Then you can just use vscode as you normally would, executing the unit tests line by line, modify the package functions and then <shift-enter> or using Revise, etc. Seamless.

7 Likes
5

I have all paths to data in tests with the pkgdir so it works interactively. e.g.

readdlm(joinpath(pkgdir(MyPackage), "test/my_data.csv"),',')'

Then the ]test works just as well as the <shift-enter> in vscode

1 Like
6

Well I got three entirely opposed answers again. Is there not a built-in way that just works? How are new users supposed to take the advice to put all their code in functions and packages if everything breaks when they do? Sorry if I sound ungrateful, but I am really frustrated with this process. Given that reproducible environments and interactive development are two of the most touted features of Julia, I am surprised by how difficult it is to get them to work. Let me ask more direct questions with full examples.

MyPackage.jl 
module MyPackage

using LinearAlgebra: norm

include("myfile.jl")

end
myfile.jl 
export f

function f(x)
    function f_sub(y)
        norm(y)
    end
    2 * f_sub(x)
end

Package Dependencies

That doesn’t sound safe. I can use packages in the REPL that aren’t in my Package/Environment? This is why I am nervous that I am testing/exploring things that will not work in production.

julia> using MyPackage

(MyPackage) pkg> status
     Project MyPackage v0.1.0
      Status `C:\Users\nboyer.AIP\.julia\dev\MyPackage\Project.toml`
  [37e2e46d] LinearAlgebra

julia> using DataFrames

(MyPackage) pkg> status
     Project MyPackage v0.1.0
      Status `C:\Users\nboyer.AIP\.julia\dev\MyPackage\Project.toml`
  [37e2e46d] LinearAlgebra

julia> df = DataFrame(x=[3,4]) # Typed
2×1 DataFrame
 Row │ x     
     │ Int64
─────┼───────
   1 │     3
   2 │     4

julia> df = DataFrame(x=[3,4]) # Ctrl + Enter
ERROR: UndefVarError: DataFrame not defined

The last line errors if I run it from file, which is the behavior I would want, but once again the REPL behaves differently if I type code vs if I Ctrl+Enter it. Seems like I should avoid typing anything except using MyPackage in the REPL.

Ensuring that I only using MyPackage fixes the ambiguity above, but then it makes interactive development impossible.

Interactive Development

julia> using MyPackage

julia> y = [3, 4]
2-element Vector{Int64}:
 3
 4

julia>         norm(y) # Ctrl+Enter
ERROR: UndefVarError: norm not defined
Stacktrace:
 [1] top-level scope
   @ REPL[3]:1

julia> norm(y) # Typed
ERROR: UndefVarError: norm not defined
Stacktrace:
 [1] top-level scope
   @ REPL[4]:1

julia> using LinearAlgebra: norm

julia> norm(y)
5.0

Because of the previous point, I think the LinearAlgebra version used here could be different from the version locked in MyPackage. I don’t know how to reconcile these first two categories. I need safety and usability.

(I have also had the opposite behavior occur where I get an error from y not being defined rather than from norm, but I can’t seem to reproduce that right now.)

Sub-Functions

julia> using MyPackage

julia> f([3,4])
10.0

julia> MyPackage.f_sub([3,4])
ERROR: UndefVarError: f_sub not defined

I could of course move my sub-function outside and before the main function, but that seems weird if the sub-function only makes sense in the context of the main function. Is there a way to test these interactively or should I really eliminate all function nesting? I could use the debugger every time I need to test sub-functions, but that is slow and comes with its own set of problems.

Debugging

I figured out while writing this question that I should be debugging from a file in the test directory rather than from the src files, which explains why my debugger wasn’t doing anything. I am still confused about how to keep the test dependencies loaded and in sync with the package dependencies though.

Testing

I don’t follow what you mean by global environment. Do you mean MyPackage, test, or 1.7?

Is TestEnv.activate() different than `Pkg.activate(“test”)?

Is the typical workflow to tinker and debug from a test/manual_testing.jl file and then to formalize that code into test/runtests.jl once it is working then?

3 Likes
7

Yes, it is very frustrating and new users cannot figure it out on their own. The core devs have pointed out before that they think test-only dependencies are imperfect for this reason. Not sure if there are plans to fix it in julia itself, bit @davidanthoff has talked about trying to fix it on the vscode side (which is necessary to support vscode testing interface).

But for your immediate needs: if you use the TestEnv I promise you that your grief will end.

For your questions, if you use TestEnv and start vscode with the project activated you never need to do any sort of changes to your paths or even register your package in (v1.7). I just clone the pacjage with vscode to my preferred location and don’t use ] dev personally, but others may have different workflows.

Sorry, meant (v.1.7) environment, The package Project.toml and the package’s test test/Project.toml should just be the normal ones that work with CI. No funny business required.

Exactly. This is the source of the issue you are having. Try it out and follow the docs. The problem with julia unit testing is basically that Pkg.activate("test") does not do the same thing that ] test does when starting a unit test. TestEnv emulates that, which means you can get a project instantiated with test-only dependencies and then code as you wish.

Typically you would have test/runtests.jl just include sub tests files. So my workflow is just to write those sub-test files while TestEnv is activated, and include them in runtests.jl if I want them to run on CI, etc.

The only difference is that maybe I put the using MyPackage for each sub-test at the top of each file rather than doing once in runtests.jl—but that is just for convenience for <shift-enter>. See https://github.com/SciML/DifferenceEquations.jl/tree/main/test for an example.

The only quirk is that if you want to make changes to the test/Project.toml to add in test-only dependencies, you need to edit that file manually (or do your ]activate test, do pacjage operations, and then deactivate it) since any package operations after TestEnv.activate() are on a temporary project file and not test/Project.toml.

2 Likes
8

I think this is by far the best alternative. I general I only use nested functions that are both very simple and, mostly, anonymous functions.

That is a fair point, but IMHO you are overly cautious. It will be uncommon that a functionality will brake between the package version that you have on Main vs. the one you are supporting with your package. That can be solved by the strategy of creating a new environment only for development suggested by @davidanthoff . Yet, I think that the most important is to put the relevant tests in the runtest file, which won’t carry any of these ambiguities, if the tests are carefully written.

9

I’m with you, I think this is all too confusing. I think there is one thing we could do in the VS Code extension that might help with this situation a lot:

If a user opens a package folder, and that package folder is not deved into the current active global project, we could automatically create the env that would be created from the ] test command (by using TestEnv.jl), and auto activate that project when a new REPL is started. So, at that point you could import/using the package that you are developing and any package that is listed as a test dependency of that package from the REPL. @jlperla, do you think that would make sense?

I am also currently working on a new test runner feature for the VS Code extension (Add test feature by davidanthoff · Pull Request #2350 · julia-vscode/julia-vscode · GitHub). That scenario doesn’t involve the REPL, but I think it should enable quite cool new dev habits when working on a package. Goal for all of that is Juliacon :slight_smile:

10 Likes
10

It would be even better if Julia itself supported some of that. Like adding dev dependencies to the environment. Could be done with

MyPackage> add Plots@dev

That would make the package available in the environment while not making it a dependency of the package.

1 Like
11

Oh, and just for the record: I think things would be a heck of a lot easier if one just couldn’t activate a package. In my ideal world, we would have a Package.toml at the root for packages that can’t be activated, and Project.toml files always go hand in hand with Manifest.toml files for projects. And the two things (packages and projects) would just be different things, and one couldn’t edit both with the package REPL etc.

I understand the appeal of the current solution in terms of reuse, and for advanced users it is probably great. BUT, I am working with a lot of new Julia users, and the level of confusion around all of this is enormous, and I think one problem is that various things in the current setup are overloaded and used for multiple roles/things. That can just be very confusing for new or occasional users.

6 Likes
12

That would be awesome and I think almost seamless for the user. I never dev anything in the global environment so I am not sure the interaction there.

The only issue is that if this is automatic then in the REPL package operations would be confusing for people (since they would apply to the temporary package/manifest rather than either the main project or test one). So I feel like it might be something that should be manually triggered (e.g. when you use the shiny new test interface) but not when just starting up the REPL.

13

Ah, that is a good point… One way out of that could be that we provide proper support for editing package Project.toml files with auto-complete etc, so that people stop using the package REPL to edit package Project.toml files…

Yeah, the new test interface is already doing that on the PR, and there is no risk for the confusion you mention because there isn’t a REPL involved in that, and therefore also no REPL package mode.

14

I have had Plots.jl updates break my code a few times now.

Does the @ in front of the environment name mean it will be separated from the 1.7 environment and actually be self-contained?

15

No, I don’t think you can have a really self-contained environment, dissociated from Main, because all the base functions are in Main. If you import a new function into Main, it will be there as well.

Concerning one specific question of the above: It is true that if you have:

julia> module Test
           const a = 2
           f(x) = 2x
       end
Main.Test

You can’t copy/paste the definition of f into the REPL to use it. I don’t know any other solution to that than to import it explicitly:

julia> import .Test: f
f
julia> f(1)
2

If you need that frequently in your workflow, I would suggest having those imports written somewhere, like # import Test: f, g, h, for non-exported functions, and use that whenever you start working with the package.

16

This approach is really great! I wonder if there is any way to integrate this with the LanguageServer.jl, so that the current environment is the one created by TestEnv.jl - I think this would solve a lot of the issues I have with the language server not working well when I am developing a package.

EDIT: When activating the TestEnv in the REPL, it gives you the directory of the temporary project, whose environment you can just switch to in VS code. It would be a nice feature if this was an option in VS code - like “Start REPL with TestEnv” which does this process automatically.

1 Like
17

I have also trained many new users on Julia and agree that this is a major pain point for them.

I wanted to add that this issue extends beyond just differentiating the package and test environments – usually a 3rd environment is needed to run interactive test scripts that will be used for development and plotting, etc. I usually advise to create an environment in the package’s /examples folder that has dev . to use the package and also has Plots or Makie added. This is an important distinction because I find that we rarely want to add plotting to the test environment, but we nearly always need plotting for demo / example scripts. For this reason, TestEnv isn’t enough for us, though it is a nice package for working on tests. The /examples environment solution does work fine for us, but nearly always causes user confusion and I’m happy to see folks are already discussing potential tooling to help with this.

9 Likes
18

For that I keep Plots installed in the main environment, effectively. And many times I develop auxiliary functions in my package which use things like:

function my_plot(d::MyDataType)
    Main.plot(...)
end

such that I can keep these testing functions in the package without requiring Plots to be a dependency.

2 Likes
19

In the past, I have seen version conflicts when I keep anything other than bare-bones (BenchmarkTools, Revise, Infiltrator, etc) in my main environment. The environment stacking solution seems nice in theory, but adding Plots to v1.7 adds a lot to that environment’s Manifest and all of those versions now have to be compatible with everything in the developed package (and all of your developed packages). When you have conflicts here, the user-facing errors can be even more disruptive than the effort of maintaining an /examples environment from what I’ve seen.

3 Likes
20

I keep the following in my startup.jl

function exportall(mod)
    for n in names(mod, all = true)
        if Base.isidentifier(n) && n ∉ (Symbol(mod), :eval)
            @eval mod export $n
        end
    end
end

and I just run exportall(MyPackage) when I introduce something new.

It’s useful for debugging a package. Though Infiltrator.jl is also super helpful.

But ultimately, OP, I don’t think it’s that complicated.

] activate . 
julia> using Revise;
julia> using MyPackage
# edit edit edit 
julia> main() # exported from MyPackage

use Infiltrator to debug.

2 Likes