Yes, Julia can be a great top language, but doesn’t have to be. I’m a great believer in languages, i.e. Julia working with others, such as Python (as C++ substitute for them), and while this seem off-topic in the discussion, it might affect installation, and package manager(s) used? I believe PythonCall.jl solves this for either Julia or Python as the top language (and pyjulia before), even for libraries of both languages.
What’s the API issue? Did you have (web) frameworks, as opposed to libraries, in mind? I believe gluing can be as good with Julia as top, except for the known TTFP/TTFX issue, that’s getting better.
Using Conda to install Julia itself is fine and already works. There seem to be some delay on that end making new versions available and sometimes their builds are misconfigured, which is why we don’t recommend it. If you use Conda to install Julia and find that is out of date or broken, please file issues with Conda.
Using Conda to install and manage Julia packages, on the other hand, definitely isn’t workable. Julia supports installing and managing multiple versions of the same package at the same time and using them from different lightweight environments. Julia also has a sophisticated system for pre-building and providing binary dependencies in a cross-platform, reproducible way that seems hard to integrate with Conda. Both of these would be very challenging to make Conda work with—the systems simply take very different approaches to installing and managing things. However, if someone from the Conda community were to teach Conda how to manage Julia’s package layout and could bridge the way the two systems manage binary dependencies, that would be great. But that work is entirely on the Conda end, not the Julia end. If you’re interested in this I would suggest opening an issue with Conda asking them to add support for Julia packages. Maybe they’ll implement it.
In the meantime, if not being able to use Conda to manage Julia packages makes Julia too much of a hassle for you, that’s ok, no one is forcing you to use it (I hope); use it if it seems worth it to you.
Julia’s environments seem similar to conda environments and Anaconda’s build service does a similar job to Julia’s, if I’m not mistaken. I would like to know more specifics about the difference between them.
For one, I can mention anaconda build system doesn’t seem to support all platforms we’re interested in (most notably FreeBSD and all musl-based Linux systems). On the other hand they support a couple of systems we don’t support (32-bit macOS and Linux on IBM Z).
This PR implements environment stacking via an opt-in configuration parameter. For now it will be undocumented. Many people think this feature is dangerous. However, as trivial as it is was now to implement, there’s no reason it can’t live behind a feature switch that users can opt in to.
Even if the differences are more minor than fundamental, there are likely to be corner cases we can’t see until someone actually tries making Conda install Julia packages or Pkg.jl install Conda packages.
100% agree with all the points you made here. Conda is a deal-breaker for me. It simply sucks to my use cases. I would not use Julia at all if conda is a hard dependency.
There is already a manual solution which is not very complicated.
It’s a bit different for Windows and Linux but it works like a charm.
EDIT: If you need to use a self-signed-certificate, you can use the same setting as for conda in the .condarc file: ssl_verify: /<path to mycerts>/cacert.pem. But additionally you have to set the environment variable MAMBA_SSL_NO_REVOKE to 1 (cf. here)
I think you may misunderstand conda: it’s just a package manager that can install packages, language agnostic, and have virtual environments. That means it can install R, C compilers, Python, and many packages (such as GPU drivers). Why not Julia?
The problem is that I don’t want to use only Julia. I also wanna use Python. And all of them in specific versions. Decoupled from the rest. How can I control that?
I mean we’re not going to submit conda-forge yaml files for Julia packages, because that’s a step in the wrong direction, Conda has showed us it is less reliable and resolves slowly and fails to capture even for Python’s very popular ecosystem (manually installing Cuda in 2022? try installing Jax + tensorflow at the same time?)
what would motivate Julia maintainers to submit entries there? when Julia’s Project.toml and registry mechanics can perfectly and efficiently solve both Package and Binary dependencies need among developer and users?
You can do that. You can use Julia as you main language and Python with PythonCall.jl (and/or PyCall.jl) and it uses Conda (or actually its clone mamba) to take care of Python dependencies.
If you really need to install Julia with Conda then it’s possible as Karpinski mentioned; and the latest (for now) 1.8.2 is there.
If you want Python to be your main language, and to call Julia, then that’s also possible, again with the same “PythonCall” package, sort of, as then with its JuliaCall Python module, and:
JuliaCall manages its Julia dependencies using JuliaPkg.
It will automatically download a suitable version of Julia if required.
In both cases you use Julia’s Pkg to install Julia packages, directly in the former case or I guess indirectly with that JuliaPkg, in that latter case.
[There’s also pyjulia, which is a redundant package, I believe JuliaCall is the future and its replacement, but still some Julia packages are callable from Python and made to use pyjulia. Since that package is related to PyCall.jl I assume all 4 packages work together if needed.]
You can also call to or from e.g R, and RCall.jl has an R REPL mode. Pkg also has a Conda REPL mode extension, I really would check out if what’s already there doesn’t work for you.
Eh, no. conda was designed as a python package manager that addressed the absolutely horrendous experience of trying to manage native code dependencies of python packages. It has indeed grown to encompass a broader range of software packages, but the thought of using it as an actual full-fledged package manager for things outside of python is so terrible that it makes me shudder. It almost sounds like you are trying to use conda to do docker’s job of handling isolated software environments.
It’s funny, every time I get a bit frustrated with waiting for Pkg to precompile things in Julia, all I have to do is remember how painfully slow, bloated, and fragile conda is and how many times I had to wait literally hours for it to resolve package environments the second I tried to configure an even slightly non-standard package environment.
Pkg is an absolute dream compared to conda, honestly.
I have mostly enjoyed using pip/virtualenv in python (never really perceived conda as better). And I enjoy using cargo for Rust, apt for Debian sysadmin, and Pkg for Julia. Each one of these tools is designed with particular goal in mind. It takes effort to learn how to use them idiomatically. But that effort pays off immensely compared to the effort of cramming one of these package managers as a solution for another language. Just do not use apt for python management or conda for julia management – it is an experience that will only cause problems. Learn the idiomatic way to do package management for the given language.
Disregarding why the idioms exist is probably why you received such a hostile set of responses
Moreover, it is indeed true that installing binary dependencies is much more difficult with pip or conda than with Pkg (I am saying this as someone that loves python and has been using it since version 2.5). This is a good enough reason to pick an idiomatic tool for Julia instead of cramming it into conda. Idioms really matter.
Lastly, of course you can simply disregard this advice and use conda or build your own extension. But for the above reasons you will probably not find many people interested in helping with your setup.
Edit: I am pretty sure you are wrong about your claim that conda has language-agnostic virtual environments. Virtualenv is a mostly python-focused notion, enabled by copying most of your python distribution to a separate folder and dynamically changing most of your PATH variables. That works ok for python, but generally, it is a pretty hacky solution with severe overhead, not supported by other languages (maybe they have crammed R into it too). Julia’s Project.toml environments have been designed with the benefit of hindsight, solving many of conda/virtualenv/pip’s pain points.
Edit 2: If you need a version multiplexer for the language, not just a package manager for the libraries in the language, juliaup is a pretty good solution, similar to rustup.
As a developer and Geo-data scientist, I’m well-versed in Python and R. While I find Conda slow, I frequently use Mamba, which is faster, though it still feels sluggish at times.
Why do I use Mamba often? It’s because I can easily install many things with it, such as MongoDB, Postgresql, PostGIS, fish shell, Open Street Map Tools, R, and R packages. I use Mamba for these tools, not for the product environment, but rather for my data analysis needs. Why not use apt, yum, or yay, or install.packages()? Mostly because Mamba offers precompiled packages, and I lack root permission on my University cluster.
Currently, I’m happy coding with rpy2 for spatial modeling. It’s great that I can install R and use Mamba on the cluster to install all the precompiled packages I require. Sometimes, I also experiment with Julia and use Juliaup from Condaforge to install Julia. I’m interested in installing Julia and Julia packages using Mamba, which is why I’m here.
From previous discussion, it seems unnecessary to install all Julia packages using Mamba, but there may be use cases for Julia. At present, the Conda-forge does not support installing Julia on Win and Mac Arm.
FYI, you can use MicroMamba (“a tiny version of the mamba package manager”) with Julia, but I believe it’s only used currently with Python, i.e. with PythonCall, or packages that rely on it.
Note, the only dependant of MicroMamba is CondaPkg, which PythonCall depends on. At some point it really meant it used Conda, but now this faster [Micro]Mamba drop-in replacement.
For Julia packages you should use only its package manager. It often gets e.g. compiled C/C++ dependencies for you (i.e. JLL packages), and while it could in theory get and install e.g. PostgreSQL for you, I believe it’s outside of its scope. Since it’s in Conda I suppose MicroMaba could be used from Julia to get it for you. Julia’s Pkg manager can and does however install embedded databases for you, e.g. SQLite (I believe also DuckDB).
What people use to install R packages, I don’t know. I suppose if you use Julia with R, you should just use what’s offered in the R ecosystem. Maybe it’s Mamba, or if you use that it could be as smooth as using Julia with Python and its package ecosystem. I don’t think it’s that smooth yet with R and Julia, i.e. you have to install, but with Julia and Python together everything can be automated.
I would use juliaup to install Julia (of any version you choose, or whatever platform, I assume Mac Arm too). In fact I just did right now for the newly released version with juliaup add 1.9.0 I do see Conda has though the older 1.8.5 (and 1.9.0 hopefully soon, though I’m not sure how important, i.e. I’ve never heard of anyone using this method to install Julia):
For Windows the main issue is that conda-forge builds on Windows are usually done via Visual Studio. We currently do not have a way to build Julia via Visual Studio.
For macOS ARM, the issue is lack of build infrastructure for that problem.
As a Python library author, I would really like to be able to have some better level of integration between conda/ mamba and Julia.
It’s not that I want to manage a julia environment with conda. It’s that I want to distribute Julia code to the python userbase. Ideally, my python package could rely on Julia code without users needing to know about it – similar to relying any C++ or Rust library available in conda. Then Julia code could reach and benefit a larger audience.
See: [PythonCall &] JuliaCall “together in seamless harmony:”
Note, PythonCall.jl has 38 (direct) dependents already (still fewer than the 229 dependents of the older PyCall): JuliaHub
E.g. PyBullet, PythonCallHelpers, PythonPlot and SQLFluff
[I don’t know where to look up dependents of Python libraries, e.g. for the JuliaCall package there (or pycall), same as I can do with juliahub, with that link.]
It’s great, it downloads Python for you if needed, all your Python dependencies, e.g. PyBullet or whatever, but I haven’t investigated as much going in the other direction. Before you used pyjulia; and PyCall, then calling to (or from) Python meant you had to mess with dependencies manually, I’m hoping, but haven’t confirmed the “seamless harmony” actually means the same in the opposite direction, what you’re doing, Python users can use their tools, and Julia dependencies just work (i.e. are downloaded automatically, and Julia itself).
