I just happen to find out about the existence of the shared environments. However, I could not find what are they useful for, and the doc string does not tell me much. What are they useful for? In what they differ from other environments?
(jl_JlkvV1) pkg> ?activate
activate
activate [--shared] path
activate --temp
Activate the environment at the given path, or the home project environment if no path
is specified. The active environment is the environment that is modified by executing
package commands. When the option --shared is given, path will be assumed to be a
directory name and searched for in the environments folders of the depots in the depot
stack. In case no such environment exists in any of the depots, it will be placed in
the first depot of the stack. Use the --temp option to create temporary environments
which are removed when the julia process is exited.
Is there any more detailed entry on the Pkg docs page? (I could not find it).
But is it special in the sense that, by any means, I can use a package from that environment in another environment (like one can use packages from @v1.7)?
I think that behavior only applies to @v1.7.
In my case, I use them as sandboxes in order to avoid cluttering my base environment.
Adding too many packages in @v1.7 puts you at risk of compat conflicts when working on projects with specific dependencies, so you may want to create various sandboxes like @optim, @machinelearning, etc.
It would work with local environments too, but these are less easily accessed.
Exactly, that is why I’m asking. I would like to have one such environment, lets say @optim, and use the packages installed there in another environment, temporary or not, so that I can use packages for testing, etc, without having to either install them in the local environment or bloating the base one.
Something like
Pkg.activate(temp=true)
Pkg.merge(@optim)
using Optim
...
What I would do in that case is take all of the packages you need in the local environment and put them in @optim, so you don’t need to activate the local env “on top of it” (since you can’t AFAIK)
The thing is that there are two types of "need"s. One is the actual dependencies of the package/script, other is the eventual necessity to use some tool (Plots, for instance) to check or test something.
yeah, I’ve been following those possibilities, and while they are a reasonable path when we are talking about a more involved package development workflow and unit testing, they do not solve the problem for more pedestrian scripts or even for the eventual use of a package or set of packages for checking something that should not persist in the development workflow in any way.
Something like
from @plots using: Plots
from @optim using: Optim, Ipopt
For that I usually have a different folder with a local environment (lets call it scripts), that has the the package dev’ed and those additional dependencies that shouldn’t be part of the project itself.
So it looks like this
Indeed, that is something that was suggested in a recent thread. Yet, that sort of implies that one would not be developing the package in the package environment. I see that is why a shared environment was suggested for that exact use. All possible, surely, but another layer of complication (and not composable either, since one still needs one of those environments for each project - sharing those would end causing the same dependency problems we may have with @v1.7).
I keep getting that answer , but I have the impression that this is a misconception about a complete set of kind of people that will use Julia. As a package developer I see the many advantages of that, but:
I don’t think it is a good thing that I have to manage environments just to sporadically plot some data which I happen to have to see while developing a package. I just want to use Plots once.
Most of the users of my packages will never develop packages. They will follow a tutorial, and get their results. Having the possibility of these users having issues with dependencies of other projects, of asking them to get used to using environments is out of question.
And I think that 2 is quite transparent to everybody here. I don’t know of any package author that suggests how to deal with environments before installing his/her package. We don’t want our users to have these barriers, so we tell them to ] add MyPackage and using MyPackage and that is it. And it should safe to do that.
edit: I consider this thread solved, my delusional speculations on this are on this other thread.
I don’t think these are mutually exclusive. What people in this thread are asking (and what I think was the intention behind environments, other than seperation of concerns) was making that whole process of e.g. following a tutorial easier. If you don’t create new environments, you’ll have packages from old tutorials laying around, messing things up. Julia gives you the tool to solve that: use a new environment for that purpose.
This of course doesn’t stop you from having e.g. Plots in your main environment, if you’re using that all the time anyway. They stack after all and if you ]activate a different environment you’ll still be able to use Plots. That’s all people are advocating for here. Saying that this workflow is “out of the question” and asking other people to work around your preferred workflow instead (even if the workflow exists to solve real problems people have!) seems a little selfish, if I’m honest.
Only because the default Julia behavior is to start in an environment that we are advocating that should not be used (that’s why I started the other thread).
Sorry if that wording felt unappropriated. It was not my intention. To be sincere, I am well aware that any suggestion I give can be dismissed for technical reasons that are way beyond my understanding, and I am ok with that. My motivation for the other thread was simply that I thought that maybe it was something that someone with the skills could read and think about it, perhaps (I hope) agree that there is something not quite ideal in the current state, and give a thought about it. It is likely that what I propose there is not a solution to anything, but something of it may be worth rescuing.
In particular, I think that the idea that Julia should start in a temporary environment with some particularly behavior that avoided these cluttering problems is something that deserved some thought. I cannot know in advance if that possibility is already exhausted.
The out of question was about telling the users of my package that they should do all that. I’m sorry, but I am the one who knows the users of the package. I do think that environments are a powerful feature of Julia. But being forced to use them (even more than one, with careful stacking considerations) is not user friendly. I think this is transparent: There is not a single package in the whole Julia ecosystem that guides the new user through the workings of environments before telling them just to use the package and have fun.
People are expected to add the packages and start following the code, like here, or here, or here, to give a few examples of important packages (examples of which I know no exception). What happens in practice is that the users naturally start bloating the base environment to find out later on that this environment thing exists. It shouldn’t be that way and, IMHO, not because the environments should be presented to the users earlier, but by making that workflow safe(r).
Would a flag for julia to start in a temporary environment work?
Something like julia --temp ?
I don’t know if that is possible, but that was a thing I was looking for in the past.
I would like to have that, would save me a few keystrokes every time. But I do not think that the current behavior of temporary environments is completely ideal, because one still needs to add the packages explicitly to them, even if the packages are available in the local machine (implying possible delays and a lot of on-screen noise). Also I think that starting in a temporary environment (with specificities) should become the default behavior, because it is for more sporadic and casual users of the ecosystem that this makes most of a difference avoiding headaches.
In the interactive REPL one does not need to add the packages manually. Typing using is enough.
Would you like to have this behavior with automatic installs also when you run a file like julia file.jl?
(@v1.7) pkg> activate --temp
Activating new project at `/tmp/jl_0ZUPqJ`
julia> using Unitful
│ Package Unitful not found, but a package named Unitful is available from a registry.
│ Install package?
│ (jl_0ZUPqJ) pkg> add Unitful
â”” (y/n) [y]: y
Updating registry at `~/.julia/registries/General`
Updating git-repo `https://github.com/JuliaRegistries/General.git`
Resolving package versions...
Updating `/tmp/jl_0ZUPqJ/Project.toml`
[1986cc42] + Unitful v1.11.0
Updating `/tmp/jl_0ZUPqJ/Manifest.toml`
[187b0558] + ConstructionBase v1.3.0
[1986cc42] + Unitful v1.11.0
[56f22d72] + Artifacts
[ade2ca70] + Dates
[8f399da3] + Libdl
[37e2e46d] + LinearAlgebra
[de0858da] + Printf
[9a3f8284] + Random
[ea8e919c] + SHA
[9e88b42a] + Serialization
[4ec0a83e] + Unicode
[e66e0078] + CompilerSupportLibraries_jll
[4536629a] + OpenBLAS_jll
[8e850b90] + libblastrampoline_jll
julia> 1u"eV" |> u"J"
1.602176634e-19 J
To me at least it’s not about never using the main environment (I use it all the time when testing a MWE from this forum, for example) but rather to only put stuff in there you know you’re using all the time. Maybe that should be mentioned more in tutorials
I agree insofar that the “teaching people about environments” part is… iffy right now for people not used to the benefits of them. Maybe this can be combined with the above and with pkg> activate @myenv to amend those tutorials, to mention something along these lines:
In julia it’s customary to use environments to seperate different projects. For this tutorial, we’re also going to use an environment, to keep anything we’re messing around with/trying out here separate from our existing projects and to not interfere with them. We can create an environment just for this tutorial by typing the following into the REPL: ] activate @ODEtutorial
This also has the advantage of being able to return to this tutorial at a later date, and resume where you left off, by activating the same @ODEtutorial again. If you’re interested, you can find out more here (insert link about detailed explanation here) about environments in julia.
Would that help alleviate the problems while also (gently) introducing the concepts?
, but I have the impression that this is a misconception about a complete set of kind of people that will use Julia. As a package developer I see the many advantages of that, but: