How to use VSCode and REPL to write and test a package?

I want the test environment to be the same as the package environment plus some carefully added extras (no stacking). This is the entire point. If I am testing in a different environment than the package environment, then I am not sure that the package will work correctly. How do I make sure that using DataFrames in my test environment is doing the same thing as using DataFrames in my package environment?

I don’t see DataFrames listed at all after TestEnv.activate().

It doesn’t look like it is a dependency in the project file on the right? Only direct packages can be accessed. See 5. Creating Packages · Pkg.jl

Maybe first get things working with ] test and then see if the TestEnv works?

Even if I duplicate all the dependencies from the package Project.toml to the test Project.toml, none of the environments use the correct version of DataFrames.

Did you pkg> resolve these environments?

Neither instantiate nor resolve changed anything in any of the environments.

Proof
(ASME_Materials) pkg> activate test
  Activating project at `C:\Users\nboyer.AIP\.julia\dev\ASME_Materials\test`

(test) pkg> status
      Status `C:\Users\nboyer.AIP\.julia\dev\ASME_Materials\test\Project.toml`
  [336ed68f] CSV v0.10.4
  [a93c6f00] DataFrames v1.3.3
  [c91e804a] Gadfly v1.3.4
  [23fbe1c1] Latexify v0.15.14
  [1986cc42] Unitful v1.11.0
  [8dfed614] Test

(test) pkg> instantiate

(test) pkg> resolve
  No Changes to `C:\Users\nboyer.AIP\.julia\dev\ASME_Materials\test\Project.toml`
  No Changes to `C:\Users\nboyer.AIP\.julia\dev\ASME_Materials\test\Manifest.toml`

(test) pkg> status
      Status `C:\Users\nboyer.AIP\.julia\dev\ASME_Materials\test\Project.toml`
  [c91e804a] Gadfly v1.3.4
  [23fbe1c1] Latexify v0.15.14
  [1986cc42] Unitful v1.11.0
  [8dfed614] Test

(test) pkg> activate .
  Activating project at `C:\Users\nboyer.AIP\.julia\dev\ASME_Materials`

(ASME_Materials) pkg> instantiate

(ASME_Materials) pkg> resolve
  No Changes to `C:\Users\nboyer.AIP\.julia\dev\ASME_Materials\Project.toml`
  No Changes to `C:\Users\nboyer.AIP\.julia\dev\ASME_Materials\Manifest.toml`

(ASME_Materials) pkg> instantiate

(ASME_Materials) pkg> status
     Project ASME_Materials v0.1.0
      Status `C:\Users\nboyer.AIP\.julia\dev\ASME_Materials\Project.toml`
  [336ed68f] CSV v0.10.4
  [a93c6f00] DataFrames v1.3.3
  [23fbe1c1] Latexify v0.15.14
  [1986cc42] Unitful v1.11.0

julia> using TestEnv; TestEnv.activate()
"C:\\Users\\nboyer.AIP\\AppData\\Local\\Temp\\jl_DrjBp3\\Project.toml"

(jl_DrjBp3) pkg> resolve
  No Changes to `C:\Users\nboyer.AIP\AppData\Local\Temp\jl_DrjBp3\Project.toml`
  No Changes to `C:\Users\nboyer.AIP\AppData\Local\Temp\jl_DrjBp3\Manifest.toml`

(jl_DrjBp3) pkg> instantiate

(jl_DrjBp3) pkg> status
      Status `C:\Users\nboyer.AIP\AppData\Local\Temp\jl_DrjBp3\Project.toml`
  [e596c8da] ASME_Materials v0.1.0 `C:\Users\nboyer.AIP\.julia\dev\ASME_Materials`
  [336ed68f] CSV v0.10.4
  [a93c6f00] DataFrames v1.3.3
  [c91e804a] Gadfly v1.3.4
  [23fbe1c1] Latexify v0.15.14
  [1986cc42] Unitful v1.11.0
  [8dfed614] Test `@stdlib/Test`

Try to (1) only put in the direct dependencies you need, and the package itself; (2) delete the manifest in the main Package folder and the one in your test folder which might be stacking in some weird way; (3) try the TestEnv again? It should then try to implement compatability bounds.

I have had some similar issues before which deleting manifests helped with. If that doesn’t work, it usually meant that I put in the bounds incorrectly.

I think this mental model is a little off and might be tripping you up here. There is no difference between them. The packages project file is used to provide a list of dependencies and constraints for usage by downstream users of the package. It is never itself instantiated (or, rather, there is no reason to and it could create garbage manifest files which trip up a workflow).

I’m arriving a bit late to the conversation, but would you have a blog post or somewhere in the documentation to recommend to setup the right workflow with VSCode ? I’ve been using Julia for the past few months and I’m getting mad at how difficult it is to have an efficient workflow (with type hints AND tests AND debugging). I parsed some information from this thread but I’m wondering if there’s some nice and high level summary somewhere.

If not, I’m especially confused here about the different environments:

  • (v.1.7): this is the “global” package env
  • MyPackage: this is where I install all the packages I need to run MyPackage. I also add packages like Revise, BenchmarkTools even though they are not specific to the project. This seems fine though according to @jlperla
  • test: this is in the end the environment I really do not understand. What do you install here? The content of MyPackage + other stuff?

Note that I never do ] dev MyPackage anywhere. Currently, I’m really annoyed that debugging either does not work properly in VSCode or that I cannot have proper type hints when writing tests, which is driving me mad.

Anyways, if you have resources you can point me to or shed some light on these different environments I’d be glad. I will start my own thread somewhere else if this is too off-topic but I felt I faced the exact same problems as @Nathan_Boyer.

1 Like

I take it you are developing a package, which you have already set up with Project.toml?
If so, just create a folder test, with the file runtests.jl. Then when you run ]test, the tests will be
executed. Look for instance at this example: Sparspak.jl/test at main · PetrKryslUCSD/Sparspak.jl · GitHub

Thanks for your answer!
Indeed I’m developing a package. I have the runtests.jl setup and I’ve been using ] test. I’m mostly struggling with (1) type completion and (2) debugging.

  1. When I write my tests with VSCode I have no type completion, which is hugely annoying. So I’m wondering if it is because I’m missing some test environment where my package is installed (if that makes sense)
  2. Regarding debugging, I tried following @jlperla advice and I have a REPL where I can do ^ + enter to have some dynamic feedback. But debugging either does not stop at my breakpoints, or simply errors in the middle of the Test.jl file when entering the macro.

Anyway, I thought that it had to do with the envs I use and if (or not) I add/dev my package in some env. Currently I never did ] add MyPackage and I’m just using the env MyPackage.

Welcome! I am also still lost on environments, especially on how to get dependencies, versioning, and namespaces to work consistently among the suggested global, package, test, and example environments. It is especially confusing because the Pkg documentation is barren of any practical advice, and PkgTemplates does not recommend creating Project.toml files the same way that Pkg does.

I have not been able to find a comprehensive guide, so I am still guessing by trial and error. Others can feel free to chime in and correct me. From what I can tell there are two main ways to run package code in the REPL:

Method 1: using MyPackage

For this method, I activate a test environment that has dev MyPackage. That allows VSCode to track changes I make to the package files. I create a file to test from that I name manual_testing.jl, and I put the file inside the MyPackage/test folder (but it could go anywhere). The first line of the file is using MyPackage. I do not using any dependencies of MyPackage. The remaining lines of the file are of the form x = 3 or y = f(x).

Pros

  1. I am testing MyPackage exactly how it would be run by others.
  2. Changes to MyPackage are implemented instantly and automatically.
  3. I can “Run and Debug” the entire manual_testing.jl file easily to find errors inside package functions.
  4. Code can be sent from the manual_testing.jl file with Ctrl+Enter or typed directly in the REPL interchangeably and reliably.

Cons

  1. I can only test full functions that are export-ed from MyPackage. I cannot test individual lines or segments of package code on their own unless I first wrap the lines in an exported function.
  2. Only function inputs and outputs are available for inspection in the VSCode Workspace.
  3. Pro #3 is really slow.

Method 2: using All, Package, Dependencies

For this method, I activate the MyPackage environment and I send code directly from MyPackage src files to the REPL. The first line I send to the REPL is using All, Package, Dependencies. I do not using MyPackage. Then I navigate to the source file I want to edit and start sending lines from that file to the REPL (in global scope). There is a VSCode command called Julia: Send Current Line or Selection to REPL that I set a keyboard shortcut for to make this easier.

Pros

  1. I can test line-by-line!
  2. Iteration on ideas is way faster.
  3. All variables are available for inspection in the VSCode Workspace.

Cons

  1. VSCode can get confused about which module I want to work in (MyPackage or Main) and error out code that should work. I typically have to choose to type all code in the REPL or send all code to the REPL from file. If I chose type in REPL, then I can copy and paste from file. If I chose send from file, then I can type a line in the file and send it.
  2. This will create a lot of temporary setup/test lines that will need to come back out for the final package.
  3. I must redefine (resend to the REPL) every time I make changes to a function. This is as easy as Ctrl+Enter, but I often forget to redefine f before executing f(x) again (since this isn’t required with the other method).

I find that I need to switch back and forth between these methods depending on if I need granularity or rigor. Method 2 Con 1 is especially annoying though; @davidanthoff any recommendations for that?

Cool, thanks a lot for this detailed answer! Will definitely give this a try on Monday.
In the meanwhile, I realized a few things:

  • We can do activate test to have a dedicated test environment (from Pkg doc). This loads the package to be tested implicitly (no need to dev it). It seems to be a good way to move the unnecessary packages (e.g. Revise, BenchmarkTools) away from the env MyPackage. Also, with Revise enabled it allows for rapid TDD by simply running ^ + enter each time I change a function in my code, until the test passes.
  • doing ] test is actually running the tests for all the dependencies of the project! Definitely slow, and not adapted to rapid TDD. So julia test/runtests.jl is not the same as ] test. Got this from a medium blog.

I will definitely understand and try your 2 methods @Nathan_Boyer, and I will try a setup with a test environment + creating a shortcut in VSCode to run runtests.jl only. One caveat I noticed yersterday and could not solve is that code completion is still not on the menu with this method (as the package is load implicitly I guess – maybe deving it in the test env should work)

Anyway, it seems unreal that setting up a proper workflow with TDD is so confusing with Julia. I will be happy to give a proper summary of all that once I worked it out.

The dependencies are checked for versions, but their tests are not run.