I’m looking for advice on a development workflow for Julia. I really like how easy it is to debug scripts (most variables can be checked in the REPL after running). On the other hand, packages with functions run much faster and their modularity is more interpretable.
Right now I:
Write everything I need in a script or set of scripts and debug, which is easy because I have access in the REPL to all the variables.
Convert scripts one by one into functions. However, when I need to change or debug them I tend to return to a script.
This workflow is somewhat tedious and doesn’t scale well. I was wondering if anyone had good alternatives. I essentially work on my own and on relatively small projects, so I’m not looking to learn a bunch of additional tools. Just want an easy way to run functions like scripts, essentially.
Hopefully this is an appropriate question; I wasn’t able to find much detail online. Thanks!
And my personal workflow specifically in respect to your questions, with a bit of advertisement
Start with a package (let’s call it MyPkg), except it is really one-off 20 lines script. PackageMaker.jl will create it comfortably for you. Creating a package may seem to be an overkill, but it isn’t. Add any necessary packages to your package.
Put/move everything into functions residing in this package as early as possible. Export each function (later on you can remove exports if a function is not to be used directly anymore).
Use scripts to make some actual calculations using these functions. Put script files within the package src (or other suitable) directory, but not include them. Put using MyPkg and other imports at the top of each script file. As soon as you need any package, add it as dependency to your MyPkg, do not install it into your main environment (or alternatively use ShareAdd.jl)
In the perfect world I’d write unit tests for each freshly defined function. But right now the test directory of my current small project is empty
P.S. If you are using VSCode, Revise.jl would be normally used by default, if it is installed into your main environment (do install it there). Otherwise put using Revise on the very top of your scripts.
Thanks for the response! I’ve come across the Modern Julia Workflows website–it is very helpful!
Your workflow makes sense, but still seems difficult to debug: I want to be able to inspect every variable after running a function, like you can do if you just run the script. Is there a way to run the REPL in the function’s context, for example?
I know this struggle well and have made several posts on this forum about it. Here are a few tips:
Make your functions smaller. A full script should probably not be converted into a single function but rather a series of smaller function calls. It is easier to check and write tests for smaller functions and smaller subsets of inputs and outputs.
The fastest method to get what you want is to temporarily write the global keyword in front of intermediate variables you want to check after the function finishes executing.
Infiltrator.jl can drop you into a REPL in the context of a local function. Then you are free to explore and manipulate variables from there.
I would strongly encourage you to stop doing that. Put all code in functions. For a “script” most of it might be in a main() function that maybe gets called at the bottom of the script. That’s already better than having the same code directly in the script (it now gets compiled!). Of course, you should still aim to refactor code into smaller function units, and eventually into packages, when appropriate. The main point is that any “global” code comes with an infinite supply of footguns and performance issues that will bite you sooner than later and cause you lots of grief.
Depending on which editor you use, another useful debug technique can be sending individual lines from a function definition to a Julia REPL. E.g., for vim-derivatives, vim-slime (what I use; there are many solutions for this kind of thing for many different editors).
Thanks all! I just played around with Infiltrator.jl and it’s exactly what I was looking for. I appreciate all the advice about workflows, performance, etc. Now time to move away from scripts…
