I want the test environment to be the same as the package environment plus some carefully added extras (no stacking). This is the entire point. If I am testing in a different environment than the package environment, then I am not sure that the package will work correctly. How do I make sure that using DataFrames in my test environment is doing the same thing as using DataFrames in my package environment?
I don’t see DataFrames listed at all after TestEnv.activate().
It doesn’t look like it is a dependency in the project file on the right? Only direct packages can be accessed. See 5. Creating Packages · Pkg.jl
Maybe first get things working with ] test and then see if the TestEnv works?
Even if I duplicate all the dependencies from the package Project.toml to the test Project.toml, none of the environments use the correct version of DataFrames.
Did you pkg> resolve these environments?
Neither instantiate nor resolve changed anything in any of the environments.
(ASME_Materials) pkg> activate test
Activating project at `C:\Users\nboyer.AIP\.julia\dev\ASME_Materials\test`
(test) pkg> status
Status `C:\Users\nboyer.AIP\.julia\dev\ASME_Materials\test\Project.toml`
[336ed68f] CSV v0.10.4
[a93c6f00] DataFrames v1.3.3
[c91e804a] Gadfly v1.3.4
[23fbe1c1] Latexify v0.15.14
[1986cc42] Unitful v1.11.0
[8dfed614] Test
(test) pkg> instantiate
(test) pkg> resolve
No Changes to `C:\Users\nboyer.AIP\.julia\dev\ASME_Materials\test\Project.toml`
No Changes to `C:\Users\nboyer.AIP\.julia\dev\ASME_Materials\test\Manifest.toml`
(test) pkg> status
Status `C:\Users\nboyer.AIP\.julia\dev\ASME_Materials\test\Project.toml`
[c91e804a] Gadfly v1.3.4
[23fbe1c1] Latexify v0.15.14
[1986cc42] Unitful v1.11.0
[8dfed614] Test
(test) pkg> activate .
Activating project at `C:\Users\nboyer.AIP\.julia\dev\ASME_Materials`
(ASME_Materials) pkg> instantiate
(ASME_Materials) pkg> resolve
No Changes to `C:\Users\nboyer.AIP\.julia\dev\ASME_Materials\Project.toml`
No Changes to `C:\Users\nboyer.AIP\.julia\dev\ASME_Materials\Manifest.toml`
(ASME_Materials) pkg> instantiate
(ASME_Materials) pkg> status
Project ASME_Materials v0.1.0
Status `C:\Users\nboyer.AIP\.julia\dev\ASME_Materials\Project.toml`
[336ed68f] CSV v0.10.4
[a93c6f00] DataFrames v1.3.3
[23fbe1c1] Latexify v0.15.14
[1986cc42] Unitful v1.11.0
julia> using TestEnv; TestEnv.activate()
"C:\\Users\\nboyer.AIP\\AppData\\Local\\Temp\\jl_DrjBp3\\Project.toml"
(jl_DrjBp3) pkg> resolve
No Changes to `C:\Users\nboyer.AIP\AppData\Local\Temp\jl_DrjBp3\Project.toml`
No Changes to `C:\Users\nboyer.AIP\AppData\Local\Temp\jl_DrjBp3\Manifest.toml`
(jl_DrjBp3) pkg> instantiate
(jl_DrjBp3) pkg> status
Status `C:\Users\nboyer.AIP\AppData\Local\Temp\jl_DrjBp3\Project.toml`
[e596c8da] ASME_Materials v0.1.0 `C:\Users\nboyer.AIP\.julia\dev\ASME_Materials`
[336ed68f] CSV v0.10.4
[a93c6f00] DataFrames v1.3.3
[c91e804a] Gadfly v1.3.4
[23fbe1c1] Latexify v0.15.14
[1986cc42] Unitful v1.11.0
[8dfed614] Test `@stdlib/Test`
Try to (1) only put in the direct dependencies you need, and the package itself; (2) delete the manifest in the main Package folder and the one in your test folder which might be stacking in some weird way; (3) try the TestEnv again? It should then try to implement compatability bounds.
I have had some similar issues before which deleting manifests helped with. If that doesn’t work, it usually meant that I put in the bounds incorrectly.
I think this mental model is a little off and might be tripping you up here. There is no difference between them. The packages project file is used to provide a list of dependencies and constraints for usage by downstream users of the package. It is never itself instantiated (or, rather, there is no reason to and it could create garbage manifest files which trip up a workflow).
I’m arriving a bit late to the conversation, but would you have a blog post or somewhere in the documentation to recommend to setup the right workflow with VSCode ? I’ve been using Julia for the past few months and I’m getting mad at how difficult it is to have an efficient workflow (with type hints AND tests AND debugging). I parsed some information from this thread but I’m wondering if there’s some nice and high level summary somewhere.
If not, I’m especially confused here about the different environments:
(v.1.7): this is the “global” package envMyPackage: this is where I install all the packages I need to run MyPackage. I also add packages like Revise, BenchmarkTools even though they are not specific to the project. This seems fine though according to @jlperlatest: this is in the end the environment I really do not understand. What do you install here? The content of MyPackage + other stuff?Note that I never do ] dev MyPackage anywhere. Currently, I’m really annoyed that debugging either does not work properly in VSCode or that I cannot have proper type hints when writing tests, which is driving me mad.
Anyways, if you have resources you can point me to or shed some light on these different environments I’d be glad. I will start my own thread somewhere else if this is too off-topic but I felt I faced the exact same problems as @Nathan_Boyer.
I take it you are developing a package, which you have already set up with Project.toml?
If so, just create a folder test, with the file runtests.jl. Then when you run ]test, the tests will be
executed. Look for instance at this example: Sparspak.jl/test at main · PetrKryslUCSD/Sparspak.jl · GitHub
Thanks for your answer!
Indeed I’m developing a package. I have the runtests.jl setup and I’ve been using ] test. I’m mostly struggling with (1) type completion and (2) debugging.
test environment where my package is installed (if that makes sense)^ + enter to have some dynamic feedback. But debugging either does not stop at my breakpoints, or simply errors in the middle of the Test.jl file when entering the macro.Anyway, I thought that it had to do with the envs I use and if (or not) I add/dev my package in some env. Currently I never did ] add MyPackage and I’m just using the env MyPackage.
Welcome! I am also still lost on environments, especially on how to get dependencies, versioning, and namespaces to work consistently among the suggested global, package, test, and example environments. It is especially confusing because the Pkg documentation is barren of any practical advice, and PkgTemplates does not recommend creating Project.toml files the same way that Pkg does.
I have not been able to find a comprehensive guide, so I am still guessing by trial and error. Others can feel free to chime in and correct me. From what I can tell there are two main ways to run package code in the REPL:
For this method, I activate a test environment that has dev MyPackage. That allows VSCode to track changes I make to the package files. I create a file to test from that I name manual_testing.jl, and I put the file inside the MyPackage/test folder (but it could go anywhere). The first line of the file is using MyPackage. I do not using any dependencies of MyPackage. The remaining lines of the file are of the form x = 3 or y = f(x).
Pros
MyPackage exactly how it would be run by others.MyPackage are implemented instantly and automatically.manual_testing.jl file easily to find errors inside package functions.manual_testing.jl file with Ctrl+Enter or typed directly in the REPL interchangeably and reliably.Cons
export-ed from MyPackage. I cannot test individual lines or segments of package code on their own unless I first wrap the lines in an exported function.For this method, I activate the MyPackage environment and I send code directly from MyPackage src files to the REPL. The first line I send to the REPL is using All, Package, Dependencies. I do not using MyPackage. Then I navigate to the source file I want to edit and start sending lines from that file to the REPL (in global scope). There is a VSCode command called Julia: Send Current Line or Selection to REPL that I set a keyboard shortcut for to make this easier.
Pros
Cons
MyPackage or Main) and error out code that should work. I typically have to choose to type all code in the REPL or send all code to the REPL from file. If I chose type in REPL, then I can copy and paste from file. If I chose send from file, then I can type a line in the file and send it.f before executing f(x) again (since this isn’t required with the other method).I find that I need to switch back and forth between these methods depending on if I need granularity or rigor. Method 2 Con 1 is especially annoying though; @davidanthoff any recommendations for that?
Cool, thanks a lot for this detailed answer! Will definitely give this a try on Monday.
In the meanwhile, I realized a few things:
activate test to have a dedicated test environment (from Pkg doc). This loads the package to be tested implicitly (no need to dev it). It seems to be a good way to move the unnecessary packages (e.g. Revise, BenchmarkTools) away from the env MyPackage. Also, with Revise enabled it allows for rapid TDD by simply running ^ + enter each time I change a function in my code, until the test passes.] test is actually running the tests for all the dependencies of the project! Definitely slow, and not adapted to rapid TDD. So julia test/runtests.jl is not the same as ] test. Got this from a medium blog.I will definitely understand and try your 2 methods @Nathan_Boyer, and I will try a setup with a test environment + creating a shortcut in VSCode to run runtests.jl only. One caveat I noticed yersterday and could not solve is that code completion is still not on the menu with this method (as the package is load implicitly I guess – maybe deving it in the test env should work)
Anyway, it seems unreal that setting up a proper workflow with TDD is so confusing with Julia. I will be happy to give a proper summary of all that once I worked it out.
The dependencies are checked for versions, but their tests are not run.
Hi! I made it here from your link on another thread.
I just to happen to be in the learning process for Julia package development as well. If I may, I’d like to contribute some things that have helped me and have mitigated the Cons you list here.
runtests.jl file, I am able to call unexported functions using MyPackageName.f_sub(x). If I need to work through writing new code, I just run those snippets in my REPL (shift+enter) from any .jl file with only using MyPackageName until I get what I want. Then I can incorporate the code into an exportable function for the API.Revise.jl loading in automatically with VSCode. As far as I can tell, this is the preferred way to develop Julia packages as it both speeds up debugging and prevents needing to redefine functions/files after changes. The way I solve this is to have Revise added in both my global environment and the project environment. VSCode sees it is added and automatically loads it when using the Start Julia REPL command in VSCode. In my opinion, this Revise functionality should be almost touted as a requirement when developing Julia packages.I really hope this helps. I admire your sustained effort in clarifying proper Julia package development techniques, and I wholeheartedly agree that there needs to be a detailed written guide on how to set this all up.
Thanks!
src requires a package dependency. For example, using MyPackage; df = DataFrame() will not work even if DataFrames.jl is a dependency of MyPackage.jl. I could using MyPackage, DataFrames; df = DataFrame(), but then I have to install DataFrames in my test environment and it could use a different version than is allowed in MyPackage’s Project.toml.src using Method 2.One option I found recently is the @run macro defined in VSCode. Writing @run f(x) in the REPL will start debugging on just that MyPackage function (as opposed to running your entire test file). It is still somewhat slow and doesn’t work on individual lines, but it works pretty well as long as your functions are broken down into small/cheap enough pieces. A limitation of this approach though is that you cannot define new variables once you are in the debugger like you could if you were in the REPL.
Also take a look at Prerelease of new testing framework and test run UI in VS Code, I’ve added yet another way to run code 
This is specifically for running tests.
This workflow suggestion doesn’t address all your questions but it does address the problem of keeping the REPL and VSCode in a consistent state.
There are a few prerequisites: Revise must be running, and your code must be in a package. When you open the REPL you have to import or using your package, e.g., using MyPackage.
Set autoSave in VSCode to onFocusChange. Now when you move your cursor from the VSCode editor window to the REPL all changed files will be saved.
Revise will detect the changes and update the REPL state. This maintains a high level of consistency between what you see in the VSCode editor and what the REPL sees.
I find this to be more convenient, faster, and less error prone than manually sending code from VSCode to the REPL using shift-Enter and similar commands.
To set VSCode to save all files on focus change open your settings.json file (ctrl-shift-P, then type Preferences: open user settings (JSON)) and add this line to the settings.json file:
"files.autoSave": "onFocusChange",
Alternatively, you can use the interactive settings UI if you don’t like editing JSON files.
Now you can edit function definitions, change variable values, etc., in the VSCode editor, and by the time you move your cursor to the REPL (in reality maybe a few seconds afterward) the REPL will have the same definitions as what you see in the VSCode editor.
We could probably think about a feature in the extension where code changes are sent to Revise without the need to save anything, that would make @brianguenter ’s suggestion more streamlined.
Eh, saving the files after making changes isn’t really my problem with Method 1. The problem is that I can’t quickly test and iterate on individual lines (which use package dependencies) within my package functions. I instead have to either:
None of these options are fast or convenient.
I guess I misunderstood. I thought you wanted to edit a line in a function in the VSCode editor, to fix a bug for example, and then immediately go to the REPL to execute the now updated function to see if it works the way you wanted. If that is what you are talking about then the settings I suggested will make the process painless and fast.
But, it seems your issue is something else. Sorry for adding to the noise.
No, it’s a good suggestion and would save some keystrokes. I was mostly directing my comment at @davidanthoff. I’d rather see VSCode work go toward better methods to test function internals than slightly faster Revising.