How to use VSCode and REPL to write and test a package?

Did you pkg> resolve these environments?

Neither instantiate nor resolve changed anything in any of the environments.

Proof
(ASME_Materials) pkg> activate test
  Activating project at `C:\Users\nboyer.AIP\.julia\dev\ASME_Materials\test`

(test) pkg> status
      Status `C:\Users\nboyer.AIP\.julia\dev\ASME_Materials\test\Project.toml`
  [336ed68f] CSV v0.10.4
  [a93c6f00] DataFrames v1.3.3
  [c91e804a] Gadfly v1.3.4
  [23fbe1c1] Latexify v0.15.14
  [1986cc42] Unitful v1.11.0
  [8dfed614] Test

(test) pkg> instantiate

(test) pkg> resolve
  No Changes to `C:\Users\nboyer.AIP\.julia\dev\ASME_Materials\test\Project.toml`
  No Changes to `C:\Users\nboyer.AIP\.julia\dev\ASME_Materials\test\Manifest.toml`

(test) pkg> status
      Status `C:\Users\nboyer.AIP\.julia\dev\ASME_Materials\test\Project.toml`
  [c91e804a] Gadfly v1.3.4
  [23fbe1c1] Latexify v0.15.14
  [1986cc42] Unitful v1.11.0
  [8dfed614] Test

(test) pkg> activate .
  Activating project at `C:\Users\nboyer.AIP\.julia\dev\ASME_Materials`

(ASME_Materials) pkg> instantiate

(ASME_Materials) pkg> resolve
  No Changes to `C:\Users\nboyer.AIP\.julia\dev\ASME_Materials\Project.toml`
  No Changes to `C:\Users\nboyer.AIP\.julia\dev\ASME_Materials\Manifest.toml`

(ASME_Materials) pkg> instantiate

(ASME_Materials) pkg> status
     Project ASME_Materials v0.1.0
      Status `C:\Users\nboyer.AIP\.julia\dev\ASME_Materials\Project.toml`
  [336ed68f] CSV v0.10.4
  [a93c6f00] DataFrames v1.3.3
  [23fbe1c1] Latexify v0.15.14
  [1986cc42] Unitful v1.11.0

julia> using TestEnv; TestEnv.activate()
"C:\\Users\\nboyer.AIP\\AppData\\Local\\Temp\\jl_DrjBp3\\Project.toml"

(jl_DrjBp3) pkg> resolve
  No Changes to `C:\Users\nboyer.AIP\AppData\Local\Temp\jl_DrjBp3\Project.toml`
  No Changes to `C:\Users\nboyer.AIP\AppData\Local\Temp\jl_DrjBp3\Manifest.toml`

(jl_DrjBp3) pkg> instantiate

(jl_DrjBp3) pkg> status
      Status `C:\Users\nboyer.AIP\AppData\Local\Temp\jl_DrjBp3\Project.toml`
  [e596c8da] ASME_Materials v0.1.0 `C:\Users\nboyer.AIP\.julia\dev\ASME_Materials`
  [336ed68f] CSV v0.10.4
  [a93c6f00] DataFrames v1.3.3
  [c91e804a] Gadfly v1.3.4
  [23fbe1c1] Latexify v0.15.14
  [1986cc42] Unitful v1.11.0
  [8dfed614] Test `@stdlib/Test`

Try to (1) only put in the direct dependencies you need, and the package itself; (2) delete the manifest in the main Package folder and the one in your test folder which might be stacking in some weird way; (3) try the TestEnv again? It should then try to implement compatability bounds.

I have had some similar issues before which deleting manifests helped with. If that doesn’t work, it usually meant that I put in the bounds incorrectly.

I think this mental model is a little off and might be tripping you up here. There is no difference between them. The packages project file is used to provide a list of dependencies and constraints for usage by downstream users of the package. It is never itself instantiated (or, rather, there is no reason to and it could create garbage manifest files which trip up a workflow).

I’m arriving a bit late to the conversation, but would you have a blog post or somewhere in the documentation to recommend to setup the right workflow with VSCode ? I’ve been using Julia for the past few months and I’m getting mad at how difficult it is to have an efficient workflow (with type hints AND tests AND debugging). I parsed some information from this thread but I’m wondering if there’s some nice and high level summary somewhere.

If not, I’m especially confused here about the different environments:

  • (v.1.7): this is the “global” package env
  • MyPackage: this is where I install all the packages I need to run MyPackage. I also add packages like Revise, BenchmarkTools even though they are not specific to the project. This seems fine though according to @jlperla
  • test: this is in the end the environment I really do not understand. What do you install here? The content of MyPackage + other stuff?

Note that I never do ] dev MyPackage anywhere. Currently, I’m really annoyed that debugging either does not work properly in VSCode or that I cannot have proper type hints when writing tests, which is driving me mad.

Anyways, if you have resources you can point me to or shed some light on these different environments I’d be glad. I will start my own thread somewhere else if this is too off-topic but I felt I faced the exact same problems as @Nathan_Boyer.

1 Like

I take it you are developing a package, which you have already set up with Project.toml?
If so, just create a folder test, with the file runtests.jl. Then when you run ]test, the tests will be
executed. Look for instance at this example: Sparspak.jl/test at main · PetrKryslUCSD/Sparspak.jl · GitHub

Thanks for your answer!
Indeed I’m developing a package. I have the runtests.jl setup and I’ve been using ] test. I’m mostly struggling with (1) type completion and (2) debugging.

  1. When I write my tests with VSCode I have no type completion, which is hugely annoying. So I’m wondering if it is because I’m missing some test environment where my package is installed (if that makes sense)
  2. Regarding debugging, I tried following @jlperla advice and I have a REPL where I can do ^ + enter to have some dynamic feedback. But debugging either does not stop at my breakpoints, or simply errors in the middle of the Test.jl file when entering the macro.

Anyway, I thought that it had to do with the envs I use and if (or not) I add/dev my package in some env. Currently I never did ] add MyPackage and I’m just using the env MyPackage.

Welcome! I am also still lost on environments, especially on how to get dependencies, versioning, and namespaces to work consistently among the suggested global, package, test, and example environments. It is especially confusing because the Pkg documentation is barren of any practical advice, and PkgTemplates does not recommend creating Project.toml files the same way that Pkg does.

I have not been able to find a comprehensive guide, so I am still guessing by trial and error. Others can feel free to chime in and correct me. From what I can tell there are two main ways to run package code in the REPL:

Method 1: using MyPackage

For this method, I activate a test environment that has dev MyPackage. That allows VSCode to track changes I make to the package files. I create a file to test from that I name manual_testing.jl, and I put the file inside the MyPackage/test folder (but it could go anywhere). The first line of the file is using MyPackage. I do not using any dependencies of MyPackage. The remaining lines of the file are of the form x = 3 or y = f(x).

Pros

  1. I am testing MyPackage exactly how it would be run by others.
  2. Changes to MyPackage are implemented instantly and automatically.
  3. I can “Run and Debug” the entire manual_testing.jl file easily to find errors inside package functions.
  4. Code can be sent from the manual_testing.jl file with Ctrl+Enter or typed directly in the REPL interchangeably and reliably.

Cons

  1. I can only test full functions that are export-ed from MyPackage. I cannot test individual lines or segments of package code on their own unless I first wrap the lines in an exported function.
  2. Only function inputs and outputs are available for inspection in the VSCode Workspace.
  3. Pro #3 is really slow.

Method 2: using All, Package, Dependencies

For this method, I activate the MyPackage environment and I send code directly from MyPackage src files to the REPL. The first line I send to the REPL is using All, Package, Dependencies. I do not using MyPackage. Then I navigate to the source file I want to edit and start sending lines from that file to the REPL (in global scope). There is a VSCode command called Julia: Send Current Line or Selection to REPL that I set a keyboard shortcut for to make this easier.

Pros

  1. I can test line-by-line!
  2. Iteration on ideas is way faster.
  3. All variables are available for inspection in the VSCode Workspace.

Cons

  1. VSCode can get confused about which module I want to work in (MyPackage or Main) and error out code that should work. I typically have to choose to type all code in the REPL or send all code to the REPL from file. If I chose type in REPL, then I can copy and paste from file. If I chose send from file, then I can type a line in the file and send it.
  2. This will create a lot of temporary setup/test lines that will need to come back out for the final package.
  3. I must redefine (resend to the REPL) every time I make changes to a function. This is as easy as Ctrl+Enter, but I often forget to redefine f before executing f(x) again (since this isn’t required with the other method).

I find that I need to switch back and forth between these methods depending on if I need granularity or rigor. Method 2 Con 1 is especially annoying though; @davidanthoff any recommendations for that?

2 Likes

Cool, thanks a lot for this detailed answer! Will definitely give this a try on Monday.
In the meanwhile, I realized a few things:

  • We can do activate test to have a dedicated test environment (from Pkg doc). This loads the package to be tested implicitly (no need to dev it). It seems to be a good way to move the unnecessary packages (e.g. Revise, BenchmarkTools) away from the env MyPackage. Also, with Revise enabled it allows for rapid TDD by simply running ^ + enter each time I change a function in my code, until the test passes.
  • doing ] test is actually running the tests for all the dependencies of the project! Definitely slow, and not adapted to rapid TDD. So julia test/runtests.jl is not the same as ] test. Got this from a medium blog.

I will definitely understand and try your 2 methods @Nathan_Boyer, and I will try a setup with a test environment + creating a shortcut in VSCode to run runtests.jl only. One caveat I noticed yersterday and could not solve is that code completion is still not on the menu with this method (as the package is load implicitly I guess – maybe deving it in the test env should work)

Anyway, it seems unreal that setting up a proper workflow with TDD is so confusing with Julia. I will be happy to give a proper summary of all that once I worked it out.

The dependencies are checked for versions, but their tests are not run.

Hi! I made it here from your link on another thread.

I just to happen to be in the learning process for Julia package development as well. If I may, I’d like to contribute some things that have helped me and have mitigated the Cons you list here.

  1. I think Method 1 you detail here is the right way to do it, as Method 2 Con 1 is a huge one, as you detail further in another post you created on this thread.
  2. I am unable to replicate Method 1 Con 1. When testing functions in my runtests.jl file, I am able to call unexported functions using MyPackageName.f_sub(x). If I need to work through writing new code, I just run those snippets in my REPL (shift+enter) from any .jl file with only using MyPackageName until I get what I want. Then I can incorporate the code into an exportable function for the API.
  3. Method 1 Con 3: From reading this post and some of your others, it seems you haven’t been able to get Revise.jl loading in automatically with VSCode. As far as I can tell, this is the preferred way to develop Julia packages as it both speeds up debugging and prevents needing to redefine functions/files after changes. The way I solve this is to have Revise added in both my global environment and the project environment. VSCode sees it is added and automatically loads it when using the Start Julia REPL command in VSCode. In my opinion, this Revise functionality should be almost touted as a requirement when developing Julia packages.

I really hope this helps. I admire your sustained effort in clarifying proper Julia package development techniques, and I wholeheartedly agree that there needs to be a detailed written guide on how to set this all up.

2 Likes

Thanks!

  1. I still find that I need Method 2 to run line-by-line.
  2. What you say is true except in the important case that the line of code you are trying to run from your package src requires a package dependency. For example, using MyPackage; df = DataFrame() will not work even if DataFrames.jl is a dependency of MyPackage.jl. I could using MyPackage, DataFrames; df = DataFrame(), but then I have to install DataFrames in my test environment and it could use a different version than is allowed in MyPackage’s Project.toml.
  3. I have Revise working like it should now. I meant that clicking “Run and Debug” on my test file runs very slow compared to “Ctrl+Enter”-ing a few choice lines in src using Method 2.

One option I found recently is the @run macro defined in VSCode. Writing @run f(x) in the REPL will start debugging on just that MyPackage function (as opposed to running your entire test file). It is still somewhat slow and doesn’t work on individual lines, but it works pretty well as long as your functions are broken down into small/cheap enough pieces. A limitation of this approach though is that you cannot define new variables once you are in the debugger like you could if you were in the REPL.

Also take a look at Prerelease of new testing framework and test run UI in VS Code, I’ve added yet another way to run code :wink: This is specifically for running tests.

2 Likes

@Nathan_Boyer,

This workflow suggestion doesn’t address all your questions but it does address the problem of keeping the REPL and VSCode in a consistent state.

There are a few prerequisites: Revise must be running, and your code must be in a package. When you open the REPL you have to import or using your package, e.g., using MyPackage.

Set autoSave in VSCode to onFocusChange. Now when you move your cursor from the VSCode editor window to the REPL all changed files will be saved.

Revise will detect the changes and update the REPL state. This maintains a high level of consistency between what you see in the VSCode editor and what the REPL sees.

I find this to be more convenient, faster, and less error prone than manually sending code from VSCode to the REPL using shift-Enter and similar commands.

To set VSCode to save all files on focus change open your settings.json file (ctrl-shift-P, then type Preferences: open user settings (JSON)) and add this line to the settings.json file:

"files.autoSave": "onFocusChange", 

Alternatively, you can use the interactive settings UI if you don’t like editing JSON files.

Now you can edit function definitions, change variable values, etc., in the VSCode editor, and by the time you move your cursor to the REPL (in reality maybe a few seconds afterward) the REPL will have the same definitions as what you see in the VSCode editor.

We could probably think about a feature in the extension where code changes are sent to Revise without the need to save anything, that would make @brianguenter ’s suggestion more streamlined.

:man_shrugging:t3: Eh, saving the files after making changes isn’t really my problem with Method 1. The problem is that I can’t quickly test and iterate on individual lines (which use package dependencies) within my package functions. I instead have to either:

  1. change the function return values to the results from the line of interest, comment out any other lines in the function that might break, call the function in the REPL and observe the output, repeat changing the line and calling the function until correct, change the function back, or
    (#3024 maybe would help?)
  2. set a breakpoint on the line of interest, run the debugger, observe the result, change the line, restart the debugger, repeat, or
    (#2500 to allow iteration within the debugger)
  3. restart my REPL session, use Method 2 until I get the line right, restart my REPL session again, start using Method 1 again.
    (#2335 to allow two parallel Julia sessions)

None of these options are fast or convenient.

@Nathan_Boyer

I guess I misunderstood. I thought you wanted to edit a line in a function in the VSCode editor, to fix a bug for example, and then immediately go to the REPL to execute the now updated function to see if it works the way you wanted. If that is what you are talking about then the settings I suggested will make the process painless and fast.

But, it seems your issue is something else. Sorry for adding to the noise.

No, it’s a good suggestion and would save some keystrokes. I was mostly directing my comment at @davidanthoff. I’d rather see VSCode work go toward better methods to test function internals than slightly faster Revising.

1 Like

For that sort of debugging like workflow I often find Infiltrator.jl to be useful.

1 Like

Is this still true? Or has there been some update? Thanks!

We now ship Revise as part of the extension for the new test feature. We don’t (yet) use that bundled version of Revise in the REPL, that still depends on the user installing it manually.

1 Like