I have been using Julia on and off for the past four years, and one of the things that I am still struggling with is the lack of a consistent workflow to develop and run standalone scripts that depend on a common series of custom packages also in state of development as well as standard packages such as PyPlot.
In Python, the developer mode in pip allows you to modify your packages after install in an existing environment without forcing you to install dependencies all over again.
Is there a similar way of achieving that in Julia that I may have missed?
Everything that I’ve seen in terms of workflows seems to be geared toward the repl, which des not work for me in this particular instance. I am aware that I can pass a flag to Julia to activate a specific environment from the command line, but as far as I know that forces you to reinstall all your dependencies in that environment.
Hello, I am not sure what you miss/need.
In the folder of your custom package you can, if you wish, create a specific environment with ] activate . and then install standard packages (] add pkgName), latest version of installed packages (] add pkgName#master) or custom, not-registered packages (] add https://github.com/yourName/YourPackage.jl.git).
This wil create a Manifest.toml and Project.toml file in the same folder of your custom script that track the packages installed.
However these are not reinstalled everytime the script runs, but only the first time. They are actually stored in the .julia folder of the user.
Only if you ship the script (and the Manifest/Project.toml) files to soneone else, these dependencies will be “installed again”…
import Pkg; Pkg.activate("."); Pkg.instantiate() won’t re-install everything every time. You can safely have this in your script without having to do a full download every time.
Thanks, what I am trying to figure out is the following:
Once my package is registered, anyone can create a script using MyCustomPackage as well as any other package available, for instance PyPlot or DataFrame. I would like to replicate this way of using the MyCustomPackage during development. The scripts are not meant to be distributed. PyPlot and Dataframe are not dependencies of MyCustomPackage, so they should not be in its dependency list. However, if I want to use them within the custom environment I am using for MyCustomPackage they have to be installed and added to the dependency list.
So I was wondering what would be the best practice. As far as I know, there are three options:
I can work within the custom environment and remove the additional dependencies once I am ready to release the package
I can install the package from a local git repo and use the dev mode to work with other packages side by side.
I can just add its path to the load path to the script and forget about environments
I was wondering if there is any recommended or standard way to go about it.
You can also keep your plotting and data wrangling packages in a separate environment that you stack on top of your MyPackage working environment so they never get mixed up.
I yet do not understand your problem. Why you do create a folder for your scripts, dev your package in an environment for this folder (i.e., have a Project.toml and Manifest.toml inside the scripts folder) and also add the other packages that are not dependencies of your package (just of the scripts) in this environment for the scripts.
may be also useful for you. It clones the package into a folder in .julia/dev and includes it into your current project.
The source may be a registered package, a repo url or a name of a package already in the dev folder.
Thanks for all the suggestions. I’ll follow the leads and see how it goes. Stacking the environments may be what makes more sense for us to minimize explicit dependencies going into the Package.
If your objective is developing a package, you could have a look also on my tutorial on the subject…
For a package you want to redistribute, take care not to check out the Manifest.toml, but only Project.toml…