It looks like even if we leave Enzyme out, the implementation (for example, the forward-pass likelihood calculation at EnzymeCon ~ Day 2 - YouTube ) is significantly faster, in this case even compared to the C++ version of the code. (With presumably better code, since the presenter mentions that they never want to go back to working on the C++ implementation.)
The talk focuses on autodiff because it’s on EnzymeCon, but if you zoom out on it, a major gain they made is avoiding the double implementation problem - that’s the original “two language problem” that Julia’s creators were talking about. The examples given at the time and in later talks by them were largely about having to prototype in one language and reimplement that same code in another language. This is the stage this project seems to have originally been in, with a Python implementation and a C++ implementation of the same thing.
Today, two-language problem is also used sometimes to refer to what are really "two-language solution"s as you put it, i.e. cases like Pandas or PyTorch. These have their issues, but they’re different from the original two-language problem.
Maybe Enzyme’s implementation details make it a two-language solution in this case (I’m not qualified to comment on that), but this is still a case of Julia solving the two-language problem (i.e. double implementation problem) as originally formulated.
I think this is a bit misleading. The repo you point to is Enzyme’s compiler internals. Enzyme takes arbitrary LLVM-based languages (Julia, Rust, Swift, C/C++, Fortran, Python, etc) and creates high-performance derivative functions. Enzyme enables users of these languages to write code in their choice of language without needing to use a different tool for derivatives (and actually enables them to even do differentiation across languages).
In that sense, Enzyme can help create solutions to the two language problem.
The Enzyme compiler itself (the repo you point to) may be written in multiple languages to provide better compatibility. In our case, we do so for a couple of reasons.
We needed to write the compiler infrastructure itself. LLVM is written in C++. We need to use C++ to use various LLVM API’s. This C++ code doesn’t make Enzyme have extreme special support for C++, but is because Enzyme had to be written in something and writing in the language of your most-used API makes the most sense. Inside of Enzyme.jl we use Julia-level LLVM ABI bindings for writing the extra Julia support. Julia itself also is written in C/C++ for the same reason (see julia/aotcompile.cpp at master · JuliaLang/julia · GitHub). Similarly, the Julia compiler/language helps solve the two language problem, but Julia’s implementation itself (for obvious bootstrapping reasons) isn’t pure Julia.
Also separately, in that repo the 57.1% LLVM code is purely part of our test suite. That repo also contains our C++ tests (and now even fortran integration tests). In contrast, our julia test suite lives in the Enzyme.jl repo (Enzyme.jl/runtests.jl at main · EnzymeAD/Enzyme.jl · GitHub)
right. So there are two things we can compare to, one is OpenBLAS, and certainly, we don’t consider using OpenBLAS a betrayal of Julia mission so to speak.
On the other hand, we often call out Numpy or Jax being inelegant.
One way is to score “how much a power user/developer need to interact with”, this way we can put OpenBLAS/MKL/CUDA driver into one camp, and Numpy-like glueing library into another camp.
I’m willing to take that Enzyme belongs to the BLAS camp, if the front-end gives you almost everything a power user/developer ever want to interact with from Julia side, and never need to understand / hook into Enzyme (C++) logic directly. I.e. if I’m making a PR to add @rule support for Enzyme.jl, do I need to understand how Enzyme itself work? I assume the answer is yes
that’s not what I meant sorry for being unclear, I was more asking “do you need to understand non-Julia enzyme in order to develope EnzymeCore.EnzymeRules in the first place”.
Because of course @rule macro is somewhat polished Julia API, but Numpy or literally any Python library with C/C++ backend can claim the same thing.
Btw I think I’m splitting hair for moot reason, I love Enzyme and what it brings to the Julia table, I just don’t think it’s a good example (primarily because it does not follow the common definition of Two language problem)
I just wanted to clarify for others that like the Julia compiler itself (being a code written in C, as well as Julia), the Enzyme compiler enables users like Paul/the EHT team to write single language applications that solve the classical two language problem.
The black hole code being an example of an application which demonstrates success on the two body problem, with Julia/Enzyme.jl enabling that.