How to use VSCode and REPL to write and test a package?

Tooling VS Code
,
33

Here is an example of a typical workflow that probably many people implement manually. I have not found much documentation describing this flow, are there better solutions? Any suggestions how to approach this more effectively are highly welcome.

Developer Alice creates a package, but neither wants to share her internal development code, nor does she want to pollute her package with the dependencies she uses for development.

Bob is an end-user, who will run the examples provided by Alice. These examples have dependencies beyond those of Alice’s package and Bob would like to have a simple way of instantiating the examples without modifying Alice’s package itself.

Alice is tasked with writing an analysis package MyPkg that she will test using BigSecretDataset. Ultimately, MyPkg will be released to the public, but BigSecretDataset won’t.

Developer Alice starts by creating MyPkg using a modified PkgTemplates. Her package will have tests, example notebooks and a separate project that she will use for development and not share with others.

This is Alice Developer’s workflow:

Alice creates a package with PkgTemplates, currently the # options are not supported:

julia> Template(interactive=true)("MyPkg")
Template keywords to customize:
[press: d=done, a=all, n=none]
 > [ ] user
   [ ] authors
   [ ] dir
   [ ] host
   [ ] julia
   [ ] plugins
# create a sub-folder examples in MyPkg package and populate with example usage
   [X] create example folder with new environment 
# create a 2nd project repo that is for the internal use of the developer
   [X] create separate project repo for development

...

PkgTemplates creates MyPkg as before, but it also creates

  • a project MyPkg_dev that uses Revise and MyPkg. MyPkg_dev is a separate folder that has its own git repo.
  • a subfolder examples in MyPkg that has its own environment, just like the tests folder.

Alice starts writing code in MyPkg/src/MyPkg.jl. Tests are added to MyPkg/test/runtests.jl, just like in any other package. tests also contains the test-only dependencies.

Interactive development takes place in MyPkg_dev. This is where she will copy BigSecretDataset and put code that is not meant for end users.

VS Code should run any interactive repl in MyPkg_dev. Probably this would need to be a manual setting in VS Code on Alice’s machine, because MyPkg_dev is not meant to be shared with the world. If we add a repl target option to Project.toml, it will point to an empty folder on user Bob’s machine.
VS Code could recognize a project MyPkg_dev and ask if this should be used as the development environment.

Alice puts a couple of simplified examples into MyPkg/examples/examples.ipynb.
Just like MyPkg/test, MyPkg/examples has its own dependencies that do not pollute the package MyPkg.

This is Bob User’s workflow:

Ultimately, MyPkg is released to the public. Bob can now:

  • pkg> add MyPkg # adds the package, nothing new
  • pkg> dev MyPkg # for making changes to dev itself
  • pkg> test MyPkg # run the tests in the test environment
  • pkg> explore MyPkg # Doesn’t exist yet: prepare a project with the right dependencies to run the examples in MyPkg

If Bob wants to just use MyPkg, he uses add, as he would today. If he wants to create a better version of MyPkg, he uses dev. Using dev is fine for development, but not for exploring the examples of MyPkg, because then Bob would need to maintain his own version of the package and manually keep track of the upstream changes. Also Bob does not want MyPkg and the examples’ dependencies in his Main environment. This is why the new command explore is needed.

explore creates a project ~bob/.julia/dev/MyPkg_explore and copies all of the examples of MyPkg/examples to this folder and adds the example dependencies.
Bob can now change the examples as he desires and still update MyPkg without merging. He won’t get any updated examples merged into his MyPkg_explore project, but that is only a minor issue and it makes sense to not overwrite Bob’s example code.

Does this make sense? Is anyone else following a similar workflow by manually creating a 2nd _dev project for each package they develop and an _explore project for each package they want to explore?

5 Likes