The role of femtolisp in Julia?



I’m working on a version of a parser using Tokenize that lints as it parses, tracks variable declarations and is round-trippable (i.e. holds whitespace/punctuation in the syntax tree) and in its early formulation it’s ~4 times as fast as the scheme version even with the extra work. I’m sure a proper attempt at speed could do better than an order of magnitude faster.


After reading through this thread, perhaps this piece of missing information for @abhi18av (and anyone else interested) is this:

If you are asking “But only - why femtolisp in particular?” the answer would be, as I understand (see this), Jeff Bezanson wrote femtolisp before beginning to work on Julia. So it was the right tool for the right person at the right time, and there has been no strong need or effort to change since.


There was a while very early on where we were using Guile (IIRC – but it could have been some other Scheme implementation) instead, on the premise that a mature, well-tested Scheme – and one that can compile to C code at that – would be better and faster. But we found that it was pretty crashy and that more time was being spent debugging Guile issues than working on Julia itself, so we switch to Femtolisp, which was totally smooth change, only 2x slower, took zero compilation time, and of course has the huge benefit that any bugs in the implementation get fixed immediately by the guy who wrote it – namely, @jeff.bezanson.

So ultimately the reasons for Femtolisp are:

  1. Scheme is excellent for writing parsers since trees (aka S-expressions) are its forte.
  2. Femtolisp is a small, simple, highly embeddable and remarkably fast Scheme.
  3. We control it (and by “we” I mean Jeff) and can fix any bugs we encounter.

Tangentally related, but we also used the Boehm collector as our GC for a while very early on but also found that to be crashy. It could have been our usage of it which was at fault, but we switched to a simple custom mark-and-sweep and that was much more reliable.


Mm hmm, thanks everyone for the wonderful resources! I’ve learned a lot from this thread :slight_smile:


That’s interesting, using Guile initially. I think that after Andy Wingo got involved, the language got a VM and wonderful S-exp based assembly. I’m really starting to grok compilers and language design.

Could you tell me more about why mark-and-sweep and not some other GC? And, I thought people prefer not having a GC so they can fine tune their applications more so how does that work out with julia.


Yes @andyferris, I did come to know about femtolisp by this femto-emacs. Only then did I become really interested it the use of femtolisp in julia.


The Tokenize.jl - I wouldn’t have found it myself! Appreciate it, @ararslan


@ScottPJones and @stevengj Regarding this thing about C++, then how does it work out with other LLVM targetting languages like Rust etc? Isn’t LLVM supposed to be a level platform to build upon.


In Julia, certain types in certain situations can be statically allocated on the stack (I think this maps to isbits-types but I’m not sure). An algorithm that doesn’t allocate any heap memory can avoid the GC.

For example:

function fib_rec(n)
    (n == zero(n) || n == one(n)) && return n

    fib_rec(n - 2) + fib_rec(n - 1)

If you benchmark this function, you’ll find it never triggers the GC, even for a composite type like Complex{Int64}.