And, let the record show that I did actually change interpreted to interactive in the tutorial before I started this discussion. I was just curious what the range of opinions might be on this topic within the Julia community. I was pretty surprised to find that there wasn’t much in terms of range. 
7 Likes
I really do think “interactive” and “REPL based” are directly addressing the usability, whereas interpreted is just utilizing a mistaken belief to get the idea across (most people think that if something is interactive it must be interpreted and can not be compiled).
So, “interactive repl based and yet compiled to machine code” does a lot to explain the situation and also invites people to challenge their often mistaken beliefs about compilation vs interpreted.
Just to add one more thing to this, I’m remaining pretty steadfast in my opinion that it is neither technically wrong to refer to Julia’s implementation as interpreted, nor is a technical definition the one that actually matters.
- It’s not technically wrong because Julia has an interpreter which is always in the background, and you can execute quite a lot of code in that interpreter, although some things need to be compiled as well. In fact, it is the presence of that interpreter that facilitates the features I’m speaking about. The interpreter is actually very important for this reason—not because it’s where your loops are running, 'cause it ain’t (usually).
- The technical distinction is the wrong one to focus on because the common usage of this term has nothing to do with execution strategy. For practical purposes, the old technical definition of “interpreted language implementation” is dead. No general purpose languages work this way. In common usage—according to my experience, “interpreted” is used to refer to languages with certain features (primarily based around runtime eval), and these languages use many different implementation strategies to to achieve this effect.
Now I’m wondering how people would feel if I started calling Julia a “runtime eval’d language with JIT compiler”. JAOT, for me, is not really related to what I want to say with “interpreted” because I’m not using the word to speak about the implementation strategy, but rather the host of features that runtime eval facilitates—including but not limited to interactivity. The shortcoming of “interpreted” is that people may infer something about implementation strategy that is not intended, so this can be a problem.
I think the notion of runtime eval really captures everything I want to say with interpreted—the only shortcoming of this verbiage is that nobody talks like this. The two likely possibilities are either
- People will think I’m simply referring to the presence of the
evaloperator, which doesn’t really capture the richness of the paradigm I’m talking about. I rarely useeval, but I frequently take advantage of Julia’s ability to evaluate code at runtime. - Other people will think it refers to the implementation strategy and think I’m talking about a line-by-line interpreter, even if I don’t use the word “interpreted”.
It seems all words are subject to misinterpretation. It’s a problem.
I think under your criteria, “interpreted” isn’t a very meaningful term, and the majority of languages commonly called “compiled” would also be considered to be interpreted. I also just disagree with associating it with a language semantically, and I think it’s best thought of as an implementation detail, or a mode of operation.
Personally, I think a clearer (but still imperfect) line to draw between compilation and interpretation is the presence or absence of optimizations which require seeing an entire function (or other code unit) and reasoning about how multiple subexpressions interact (i.e. an optimizing compiler). That is, an interpreter is going to just sequentially go from expression to expression (whether or not it’s been lowered to another form of byte-code) and execute them, whereas a language with an optimizing compiler will look at large code units and make substitutions and optimizations that require information about the whole function or other code-unit.
I think this is both a more useful deliniation between the terms, and also much more closely matches how the term is actually used by people in the real world.
Could you elaborate then on what you mean here by ‘runtime eval’? What do you mean by evaluate code at runtime? To me, that just sounds like “the program actually computes something at runtime, instead of compiling away the entire execution”, but I assume that’s not actually what you mean.
1 Like
What I mean with runtime eval is that Julia code is parsed and compiled at runtime, and no matter what stage of compilation you’re at, you can always generate/parse new Julia code (and yes, I realize this mostly requires going back to the top level to update the “world age” or whatever). I may have just summed a million numbers in a loop with tight machine code and SIMD, but I can directly go on and add new methods to the sum function. I can define a new macro and generate more code while my program is running. I can hop into a REPL and extend an existing library interactively. I can load a source file in the middle of program execution.
A Julia process (at least at the top level) is always ready to accept and integrate new code into the program, interactively or not. Julia is not unique in this regard. Most so-called interpreted languages have this capability.
I seek a word to describe this, and “interpreted” is what first comes to mind.
Ah I see. Yeah, to me that really has little to do with “compiled” vs “interpreted” and more to do with interactivity. I would say something like “julia is a highly interactive and responsive language to use”. If you want to talk technical features, I’d say “It has a fantastic REPL, notebook support, and strong support for hot-code-reloading” or something like that
1 Like
Would you agree that, in the current implementation of Julia, it is the interpreter that enables the features I am speaking about?
Interesting question! I don’t actually have a very strong opinion on that. If you pressed me though, I’d be tempted to say that it’s kinda the opposite – my understanding of our interpreter is that it was cobbled together using the tools which were in place to achieve those above interactive features, not the other way around.
But I’m not an expert on the implementation details, nor the history.
But I’m pretty sure we’ve had those features longer than we’d had what I’d call and interpreted mode
I believe interpreter.c is older than the interpreted mode of execution being an option for the user. I believe Julia was designed from the beginning with the intention of the interpreter and compiler working together, even if the interpreter is “only” used for top level eval, but I must admit that I don’t know the details of how things have been in the past in that regard, and perhaps I am mistaken.
My opinion is that the top level interpreter is not “just” an insignificant implementation detail, because it enables a lot of powerful features, even if most code isn’t executed there.
It may be that this is based on a flawed understanding of the Julia implementation, but it seems like Julia’s developer documentation corroborates my understanding.
It is great then that Julia is upholding the grand LISP tradition of defying conventional classifications and generally confusing the heck out of outsiders.
4 Likes
Yeah that’s not an unfair perspective. I guess the thing for me is just that what makes julia julia is that it’s not just another interpreted language and it’s taking a very different approach from traditional interpreted languages.
There’s very strong elements from both compiled and interpreted languages present, but I feel it’s more like a compiled language that’s built to feel like an interpreted one. But you’re of course not wrong to say there’s elements of an interpreter there (though id mention that interpretation of top-level forms and then compiling everything inside is hardly unique to julia and is present in most ‘compiled’ languages).
One of the main points I don’t agree with is, that just because Python becomes less of a classically interpreted language, the meaning of interpreted should change.
As much, as a yellow post car turning grey with age shouldn’t change the definition of “yellow” as a color.
Besides, I do think that CPython still satisfies the textbook definition pretty well, since the definition does includes various transformations into bytecode, and the main point is that at the bottom of the execution stack you don’t create binaries, but instead have a binary which executes bytecode/statements step by step.
The other point I don’t agree with is, that Julia’s interpreter is strictly necessary to enable the interactive REPL workflow.
Runtime eval and adding new methods is not implemented via Julia’s interpreter. It’s implemented via LLVMs JIT which allows mutation of the method table.
That eval does use the interpreter is mainly because it can contain statements running in global scope.
To be fair, I can be wrong here, because its not well documented, and I only remember this from conversions from quite some time ago, but the interpreter is just the easiest way to implement running statements in global scope.
You can also compile all statements in global scope and implement global scope like that, but then you would need to do pretty annoying shenanigans to implement a different scoping in the compilation unit - and it also doesn’t make a lot of sense to compile this, since that code will only run one time.
So a super simple 800 LOC interpreter is the easiest way to implement this, but is not strictly necessary for Julia’s interactive workflow, turning it into an implementation detail.
Just to demonstrate that you could implement an interactive, global scope with Julia’s JIT compiling everything into native functions:
# const global_scope = Dict{Symbol, Any}()
a = 22 # Invokes JIT to compile: (global_scope)-> global_scope[:a] = 22
b = a + 33 # compiles to: (global_scope)-> global_scope[:b] = global_scope[:a] + 33
You may say, oh, the library invoking the compilation is just an interpreter, and honestly at that point I’d give up arguing 
And to be fair, after reading the wikipedia page about interpreters again, they do list JIT as something interpreters use, so your statement of “JIT is just another way to implement an interpreter” doesn’t seem as wrong as I initially thought (Which comes from the wikipedia article on JIT compilation trying to differentiate itself from interpretation pretty clearly).
So, I guess strictly speaking, one can call Julia interpreted 
But with every typical Julia program spending 99% of the time in natively compiled binaries, I find it hard to see what you get out of calling Julia interpreted while there are better words for the feature you want to describe.
Also most audience I’ve been talking to inside and outside the Julia community most strongly associate interpreted with no compile times and slower runtime speed, and a good REPL experience is not necessarily a feature of an interpreted language, but comes as a separate program.
I had quite a few very heated discussions with C++ programmers who’ve never touched Julia, who had the strong impression that Julia is interpreted and therefore cant reach native performance too many times, to find it productive to call Julia interpreted.
But, if you have the opposite experience with your audience, I guess nobody can deny you that experience
3 Likes
One thing worth mentioning is that JIT isnt the most useful term here because Julia’s JIT strategy is rather different from most JIT compilers out there. Typical JIT compilers work by interpreting the code, and recording how often specific code paths get hit, and then compiling the paths which get hit a lot. This can do things like specialize on runtime values and only compile subsections of functions.
To me, that’s much more in the realm of sprinkling some optimizations on top of a fundamentally interpreted language/implementation, whereas julia’s approach is very much compilation-forward.
6 Likes
New hot take: C is an interpreted language since the compiler is typically invoked from the shell which is usually interpreted.
2 Likes
I’m not sure this is true, but I think at this point we actually need to the input of someone who knows Julia’s implementation better than either of us. I was under the impression the interpreter does play a role in runtime eval, but I guess we need more information about this topic.
I do remember there was a point in the past when you couldn’t use global variables in loops at the top level in the REPL and they changed some things to make this possible because people found it pretty weird.
Traces of this can still be found when trying to do the same thing in a script, which isn’t something that should be to common in real code:
#!/usr/bin/env julia
x = 0
for i in 1:10
x += 1
end
println(x)
.
$ ./loop.jl
┌ Warning: Assignment to `x` in soft scope is ambiguous because a global variable by the same name exists: `x` will be treated as a new local. Disambiguate by using `local x` to suppress this warning or `global x` to assign to the existing global variable.
└ @ ~/loop.jl:6
ERROR: LoadError: UndefVarError: `x` not defined
Stacktrace:
[1] top-level scope
@ ~/loop.jl:6
in expression starting at /home/ninjaaron/loop.jl:5
vs in the REPL
julia> x = 0
0
julia> for _ in 1:10
x += 1
end
julia> println(x)
10
However, I think this is not solely related to the use of the interpreter, since the interpreter is invoked every time Julia runs, not only in REPL mode.
Runtime eval and adding new methods is not implemented via Julia’s interpreter. It’s implemented via LLVMs JIT which allows mutation of the method table.
This is not really accurate, at least if I understand your meaning correctly. The last time I heard, which was around Julia 1.4 days, the method table and method resolution was managed in a C runtime library (this could be considered part of the interpreter, in fact). Once the right method is found, it either sends it off to LLVM for code-gen or used cached machine code. LLVM itself has nothing to do with binding methods to functions or method resolution, last I heard, which was a while ago.
I actually remember looking at the code for method resolution back then during a talk at JuliaCon which was based on a thesis demonstrating Julia was “soundly typed”.
So a super simple 800 LOC interpreter is the easiest way to implement this, but is not strictly necessary for Julia’s interactive workflow, turning it into an implementation detail.
I would not restrict the interpreter to only what is found in interpreter.c, but also include toplevel.c and the other runtime C libraries it depends upon. julia is a (mostly) C program which executes Julia programs—mainly by generating machine code with LLVM—but there’s still quite a lot going on in the Julia runtime.
Historically, interpreted and compiled were
- disjoint sets,
- with clear performance implications.
Neither of these have been valid claims for a while now, but I expect that programmers who specialized to older languages and don’t explore alternatives will continue to believe them.
I agree with you about these labels being unhelpful. Categorizing Julia with either of these terms is pretty much irrelevant and degenerates to an exercise in pedantry. OTOH, explaining how Julia works can be informative.
7 Likes
For some more interesting history (which I was only peripherally following so the rest is my understanding and shouldn’t be taken as gospel), SBCL came about because CMUCL was hard to maintain. The build process of CMUCL involved bootstrapping the code by compiling it using itself until the output reached a fixed point (or something approximately like that). SBCL people decided that it was better to have a system where you run the code through any common lisp (including potentially SBCL) once and got a consistent output. For various reasons they also decided to rip the interpreter out of CMUCL. So while CMUCL dealt with latency issues by having an interpreter and then compiling the important stuff, SBCL was more like Julia in that “everything” is compiled (and there are some latency issues relative to CMUCL).
So, can you rip the interpreter out and have just a compiler and still get interactive REPL based workflow? Yes definitively because SBCL did it. Julia’s interpreter is I think very much an implementation detail. The semantics of what Julia code means are unaffected by whether you interpret the code or compile it. Almost all of the code you’ll ever run is compiled… It’s a compiled language with an ancillary interpreter to make it easier to do some stuff.
This whole discussion is really just yet another foray into a long history of what I consider to be a settled question that won’t die (a vampire question so to speak). Essentially all Julia code results in the production of x86-64 (or ARM or RISC-V or whatever) binary code that is executed by telling the processor to jump to that location in memory and execute, so Julia is compiled by virtually everyone’s definition of what it means to compile. You start with high level code, and you wind up with machine instructions.
3 Likes
What you are describing here are the qualities of a “dynamic programming language”. This means the language can add new code or even modify prior code. Much of Julia’s dynamic qualities can be traced to inspiration from Lisp predecessors.
This should not be confused with dynamic typing or dynamic programing.
The opposite paradigm is a “static programming language”. Here we can make assumptions that no further changes will occur. This simplifies analysis of the program. Even Julia has elements of a static language in that we compile some functions as if nothing will change. If the assumptions do change, then we must invalidate the prior compilation and recompile the function. Over time, Julia is gaining some static programming language features.
“Static” vs “dynamic” programming languages is an orthogonal property to the execution mode which could be “interpreted”, “compiled”, or something in between.
The primary unit of compilation in Julia is the “method,” a variant of a function for the types of its arguments. Because of this much of the code outside functions is not compiled. Recently this has started to change as we have introduced package images and incremental system images.
While much of Julia’s speed is attributed to the compilation execution model, this also is responsible for latency resulting in long times to first execution.
Julia could be interpreted. One example of this is the wasm demo for Pluto.jl.
Quoting @fonsp from the issue related to the demo, “Julia feels faster because it is interpreted, not JIT compiled.”.
In summary, I think you may have conflated “interpreted” with “dynamic”. “interpreted” describes the mode of execution but is not necessarily intrinsic to a language. Julia is currently mostly executed via “compiled” functions resulting in both its speed but also its latency. However, Julia could be interpreted, compromising execution speed for potentially less latency. On the other hand, “dynamic” refers to the ability of new code to modify the execution of old code permitting the flexibility that you are assigning to “interpreted” languages. Julia may seem unusual to some because it is “dynamic” and “compiled”, although Julia follows from a long line of such languages, notably the Lisps.
5 Likes
Your example there is due to “softscope”. This can be removed from the REPL by removing one of the Abstract Syntax Tree (AST) transforms.
julia> Base.active_repl_backend.ast_transforms
1-element Vector{Any}:
softscope (generic function with 1 method)
julia> Base.active_repl_backend.ast_transforms |> empty!
Any[]
julia> x = 0
0
julia> for _ in 1:10
x += 1
end
┌ Warning: Assignment to `x` in soft scope is ambiguous because a global variable by the same name exists: `x` will be treated as a new local. Disambiguate by using `local x` to suppress this warning or `global x` to assign to the existing global variable.
└ @ REPL[4]:2
ERROR: UndefVarError: `x` not defined
Stacktrace:
[1] top-level scope
@ ./REPL[4]:2
You are confusing the “runtime” with the “interpreter”. The runtime is just code that runs with or along side of your program. Notably the garbage collector in many languages runs as part of the “runtime” and is independent of whether a language is interpreted or compiled.
There are also several ways to execute Julia code. Not all of those ways goes through the interpreter to invoke functions at the top level. For example, you could use @ccallable to create a C callable function pointer that can be executed directly by a C program. For example see libcg.
Another mode is to fully statically compile Julia, in this case without executing the runtime:
At the end of the day, “calling Julia interpreted” does not quite make sense. Whether Julia is compiled or interpreted is not an intrinsic part of the langauge itself. It describes how you execute the a program in the language and there are many ways to do so. Largely at the moment, some Julia is compiled at some point on the path to execution but that may not always be the case.
1 Like
From a pedagogical perspective, this is a quite interesting question. Thanks for posting it and prompting the discussion. I’ve personally learnt a lot by reading this thread.
In my opinion, one of a fastest ways to be productive with a programming language (or any tool for that matter) is by understanding how it works under the hood. But usually documentation and tutorials have to build on existing knowledge or context, so introducing new concepts is done at a later stage or not done at all.
I was curious how other languages handle a discussion about how a language works under the hood. As an exercise, I spent a few minutes looking at the documentation of other languages that I’ve never looked into before that have some similar properties as Julia (interactive REPL, performance, compilation, etc).
Clojure:
From Clojure - Dynamic Development, here’s a screenshot
Some quotes that stood out on different pages in the documentation:
Clojure is a compiled language, so one might wonder when you have to run the compiler. You don’t. Anything you enter into the REPL or load using load-file is automatically compiled to JVM bytecode on the fly. Compiling ahead-of-time is also possible, but not required.
If an expression needs to be compiled, it will be. There is no separate compilation step, nor any need to worry that a function you have defined is being interpreted. Clojure has no interpreter.
Clojure’s documentation even has a language on evaluation: Clojure - Evaluation
Scala
From https://docs.scala-lang.org/scala3/book/scala-features.html:
It’s statically-typed (but feels dynamic)
It runs on the JVM (and in the browser)
It interacts seamlessly with Java code
In interactive mode, the REPL reads expressions at the prompt, wraps them in an executable template, and then compiles and executes the result.
This section also provided more information:
https://docs.scala-lang.org/scala3/book/scala-features.html#a-dynamic-feel
OCaml
Separate compilation of standalone applications: Portable bytecode compilers make it possible to create stand-alone applications out of OCaml programs.
Efficient compiler, efficient compiled code: OCaml offers two batch compilers: a bytecode compiler and a native code compiler. Bytecode compilers quickly generate small, portable executables. The native code compiler is slower, but it produces more efficient machine code, whose performance meets the highest standards of modern compilers.
OCaml code can also be compiled separately and executed non-interactively using the batch compilers
ocamlcandocamlopt.
All three of these languages have REPLs.
If I were completely new to programming languages, and didn’t know anything about intepreters or compilers, in all three languages above and in Julia, I could open a REPL and type in some code, hit enter, and expect that code to run. This “feature” seems to be by and large referred to as “interactive”, and is not dependent on whether it is interpreted or compiled, dynamic or static, etc.
I do think learning that Julia compiles functions to LLVM code just before executing them (what we are referring to as “Just Ahead Of Time”) while running the code is a very unique feature in the programming landscape. Maybe it should be called “Just Ahead of Run Time Compiled” code?
And, I personally think understanding this is important to being able to take advantage of what Julia offers. But I’m still not sure how or when is best to convey that in introductory tutorials 
3 Likes