Package management via conda?

I currently use python and R for data science tasks. I’m considering trying out Julia, but I’d like to manage all packages (python, R, and Julia) via conda. Is that possible? I see that Julia can be easily installed via conda, but all instructions on Julia package installation state to use the REPL. I can’t find much info on using Julia with Conda, except for the Conda.jl package.

3 Likes

Well, this is not going to work. Julia has its own package manager which is very good, integrated and obligatory for using Julia packages. It uses Conda to manage Python packages that you might want to use from your Julia programs (if any).

And please do not install Julia using Conda for productive use, you will not get much support if you run into problems.

For using R from Julia you can use RCall.jl; Installation · RCall.jl

But this assumes you have Julia and R installed independently.

6 Likes

If this is the accurate link, Julia :: Anaconda.org, you will also get the quite outdated Julia 1.1.1, which misses out on two years of rather substantial Julia improvements.

4 Likes

Thanks for the info! So, just to be clear, I guess one could have to use Docker or another container system to fully create/manage an environment containing Julia + R + python + command line software, right?

No you can use Conda for R, python and julia downloaded from its homepage together.
If you need docker for another reason, this works of course, too.

2 Likes

It’s great that Julia has it’s own manager, which is quite solid, but it’s not the same as what conda does: it manages multiple languages.

So say I want to install a specific version of Julia with Python (in an virtual environment), how would that work?

1 Like

From a conda prompt:

exit
pacman -Ss julia
1 Like

Yes, I also am interested if it’s possible to create an isolated conda environment with julia as the base, as the default language when opening jupyter notebook, and then be able to run python packages as well.

1 Like

just use Julia and use Conda.jl

1 Like

Maybe not quite what you’re after, but PythonCall.jl (and its companion Python package juliacall) manage dependencies for you.

That is, PythonCall makes a conda environment inside your Julia project and uses that to manage Python package versions (including Python itself).

Conversely juliacall makes a Julia environment specific to your Python environment and uses that to manage Julia package versions (and uses JILL to manage Julia itself).

Both packages have ways to update these environments at the REPL or via static config files.

3 Likes

Maybe I misunderstand the answer, but I don’t think that is even close to what I was looking for. This will, on a system with pacman install julia via it. But I would like to stay inside the conda environment and possibly install julia with conda in order to keep different versions of julia

Say I have an environment with Python 3.n, and I would like to install Julia 1.m as well. I can do that with conda, but it seems that for example jupyter notebooks don’t work with julia then and

Wanting to manage more than one version of julia is fine, but be aware that the binaries provided through conda are not officially distributed and may mess with the way some things (like which libraries are loaded) work.

Even in the version provided by conda, package management is not integrated with it. Wrapped python dependencies are managed through a seperate conda environment, in order to not mess with system installations (I think there was a switch to use the system python/conda instead, but since different systems have different system-wide python installations, this is not as reliable as managing it explicitly via Pkg).

That being said, different julia versions can live next to each other without interacting with each other. You can “just” have a directory with the julia 1.7 binary, as well as a different directory with a julia 1.6 binary etc. and launch whichever you feel like. There are also some experimental tools that help you manage that process, like juliaup (inspired by rustup).

1 Like

It’s great that Julia has it’s own manager, which is quite solid, but it’s not the same as what conda does: it manages multiple languages.

…and this is why I still avoid julia software. I can easily install & manage 1000’s of R, python, and other data analysis software via conda (conda-forge & bioconda). All of the software can be listed in a simple yaml file. So why bother with julia? I don’t see why julia is worth the extra hassle. Just because “Julia has it’s own manager, which is quite solid” is not a great reason to move away from the multi-language support of conda (and huge community support).

Well, Julia solves the multi-language problem: If you use Julia you do not need any other programming language. It is your choice: Use conda, Python, R etc, or use Julia. You will not need both. :slight_smile:

2 Likes

Well, I do think pythoncall/condapkg does solve what you are looking for. CondaPkg specifically lets you
integrate conda packages into Pkg.jl fairly seemlessly.

3 Likes

That’s an ambitious claim :wink: And one that I would definitely disagree with: for the sake of performance and predictability, Julia is nowhere as flexible as Python and therefore makes gluing and API a lot harder IMHO.
I think it’s actually what harms Julia a bit: seeing itself as the most upper language. I see it rather as a great alternative to C++ for scientific computing, but maybe I am also lacking Julia experience here.

Main point: IMHO, too often languages and libraries aim to be “the top, controlling one”. Why can’t this be left to someone else, if even? It’s great if julia can manage it’s own packages, that’s not bad. But the Julia version itself, the root, should be simply installable with something else, allowing to have multiple julias installed.

This IS currently a deal-breaker for me: I don’t see an easy way of having multiple Julias installed and switching between them (no terminal hacks).

1 Like
3 Likes

I don’t see an easy way of having multiple Julias installed and switching between them

I use GitHub - johnnychen94/jill.py: A cross-platform installer for the Julia programming language for this purpose. Works very well for me (on LInux).

1 Like

There are good reasons for Julia to use its own package manager rather than tethering its fate to Conda. It would be an opinionated and heavyweight dependency, and not everyone thinks that Conda is the bees knees. Being able to drop it is viewed as a considerable advantage by many, particularly since Julia’s package manager is very nicely designed and works well.

Furthermore, Conda is not really intended as a robust multi-language solution. It’s a Python thing which later had R support tacked on. I did my R installation via Conda a few years ago, and when I ran into some weird package installation errors, I came across some advice from Hadley Wickham to install R from the R website instead, because Conda-specific bugs are quite common. Which is what I’ve done ever since.

None of this is specific to Julia. There are a dozens of new programming languages out there, and not one of them has chosen to subordinate itself to Conda. They’re all facing similar considerations, and it’s such a straightforward call that I doubt many have even contemplated it.

My advice is to use tools the way they are used by other people. If you want to learn Rust, use Cargo. If you want to learn Haskell, use Cabal. Trying to fit a tool into an idiosyncratic workflow is a recipe for a lot of work and frustration for very little gain. This is a lesson I have learned the hard way.

If you don’t like that advice and are fully committed to Conda, that’s fine. Python is a big world, and lots of people can get their stuff done without leaving it. But while I can’t speak for the Julia community, I can make a very high-confidence guess that the plea in this thread won’t get a lot of traction. Moving to Conda would have numerous high costs and offer little advantage, since the vast majority of users are fine with learning a new package manager when they learn a new language. And even if they weren’t, only a subset of Python users use Conda, so switching wouldn’t even solve that problem.

29 Likes

One more bit about juliaup: it is not yet fully done on non Windows platforms, but it will become the official recommended way to install Julia soon ™, once we have ironed out a few more things.

7 Likes