Scala and OCaml do not have the properties of Julia that I’m speaking about, though Clojure does.
I specifically know OCaml quite well and programs which are compiled to machine code don’t have these dynamic elements. The OCaml REPL (top-level, in OCaml parlance) is essential a different implementation than the native compiler, though they share a lot of common infrastructure. Unlike Julia, the REPL is based on a bytecode interpreter.
Of course the JVM is also a bytecode interpreter, but if you’re using Hotspot, some bytecode will also be JIT compiled to native after profiling.
However, OCaml’s parser and compiler is not available in the native execution environment. There is no runtime eval of OCaml code in the compiled binaries. I believe this is also the case with Scala, but I don’t know it well.
Clojure, on the other hand, does have the Clojure parser and bytecode compiler available at runtime. It has to. It’s a Lisp.
This is really the quality that interested me. Having the parser and compiler in the runtime.
And, to return to another reply’s point, unlike SBCL, Julia’s main implementation doesn’t accomplish this by bundling the compiler into compiled programs, but by running Julia code through a C program that generates native code at runtime—a C program which runs all top-level code though an interpreter which decides if it will be compiled to native or not. (and almost always does compile to native)
Julia 1.9 not only has an ahead-of-time compiled system image generated from Julia code but also precompiled package images also generated from Julia code. Both are native shared libraries (.so files on Linux). The results of top-level code executing are cached as well as the native code generated to execute them.
I am very far from an expert here. But my impression is the structure is more like:
scheme based parser (soon to be replaced by JuliaSyntax)
Parsed Julia → Lowered Julia (written in scheme and Julia)
Lowered Julia → LLVM IR compilation (written in C)
LLVM → Native code (the LLVM library written in C++)
Allocator, GC, libUV, i/o, threads and certain other low level features (written in C).
For actual interpreting of code in quantity, mainly for debugging purposes, A Julia interpreter and debugger describes JuliaInterpreter.jl which is used in the debugger and written in Julia.
I suspect the effort to move to JuliaSyntax will result in (1) and (2) implemented in Julia.
At the moment generating LLVM IR is in C but it could potentially be written in Julia in the future?
LLVM is going to stay in C++ obviously.
The runtime support stuff is going to stay in C, though I think some of the allocator for Arrays etc is moving to Julia soonish?
Basically, implementation details aside, Julia is most like SBCL or Clojure than any other language as far as I can tell. The implementation detail that some of the code is written in C is just not that relevant. For example GNU Common Lisp compiles Common Lisp to C and then invokes a C compiler to generate native code and links it into the running system, but it’s pretty clearly a compiler not an interpreter.
Very small correction, but LLVM IR generation from optimized Julia IR is in c++ and so are the custom LLVM passes and interface. But the rest is correct.
This is more a question of self-hosting or not. SBCL, despite having a self-hosting compiler still has C runtime libraries, like most garbage collected languages. (Go is an exception to this.Go famously doesn’t even rely on libc for its runtime—only for a few optional standard library packages. This has to do with their goal of being able to cross-compile statically linked binaries that will run anywhere, but this is largely irrelevant for a language like Julia unless they eventually want to focus more on binary executables)
Whether method resolution should be considered part of the runtime or the interpreter isn’t a super meaningful distinction in any case. Everything Julia does is part of the runtime, somehow. Code cashing and linking of cached code happens in the runtime. Compilation happens in the runtime. What’s more, almost everything that happens at runtime, outside of some setup and tear-down, is invoked from within the context of the top-level interpreter.
It’s an implementation detail, sure, but it’s one that makes sense because it’s the simplest way to accomplish a lot of what they want to do. If Julia ever wants to move more in the direction of binary executables, they would have two options (that I can see):
Bundle LLVM into the executable for runtime codegen.
Change some of the semantics and remove at least one feature from the language. The eval operator needs codegen, and the semantics of something like include would change slightly because code would be sourced at compile-time rather than when the program is executing, though this change is not likely to bother many people.
SBCL takes the first option. Though it doesn’t rely on something as large as LLVM for codegen, you still can see the bundled compiler in the size of the binary. Clojure… well, the Clojure->bytcode compiler is in the compiled artifact as well, but this is a much smaller cost, since the JVM is doing most of the work in terms of optimization and native code generation.
In Julia’s case, I’d probably rather see the second option, so long as the current implementation with interpreter was also available.