# Roadmap for small binaries

**URL:** <https://discourse.julialang.org/t/roadmap-for-small-binaries/99266>\
**Category:** Internals & Design\
**Tags:** question\
**Created:** [May 23, 2023, 8:41am UTC](https://discourse.julialang.org/t/roadmap-for-small-binaries/99266 "2023-05-23T08:41:09Z")\
**Posts on this page:** 20\
**Page:** 4

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [October 22, 2023, 8:40pm UTC](https://discourse.julialang.org/t/roadmap-for-small-binaries/99266/61 "2023-10-22T20:40:49Z")

</div>

> [@TheLateKronos](#):
>
> I could be wrong, but could we not compile inefficient compiled code for abstract types, e.g. Number? Compiling for a concrete type is only needed for maximal performance, no?

This depends on the definition of “small”. This compiled code would need to dynamically dispatch which might require the Julia runtime. When some people mention “small” they mean self-contained without linking any Julia runtime. If we can link the runtime, then this is possible as far as I understand.

We really need better language to describe the different kinds of binaries that people want.

---

<div class="post-metadata">

**Author:** ![joa-quim](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/joa-quim/32/227_2.png) [@joa-quim](https://discourse.julialang.org/u/joa-quim)\
**Post date:** [October 22, 2023, 8:42pm UTC](https://discourse.julialang.org/t/roadmap-for-small-binaries/99266/62 "2023-10-22T20:42:34Z")

</div>

I think people are forgetting another part of the story here. The size of the pre-compiled cache files. They are (relatively speaking) huge files. Makie is 130 Mb. GMT (the one that bothers me more for direct reasons) is \> 100 Mb on Julia nightly. Why are those sooo big?

---

<div class="post-metadata">

**Author:** ![greatpet](https://avatars.discourse-cdn.com/v4/letter/g/e495f1/32.png) [@greatpet](https://discourse.julialang.org/u/greatpet)\
**Post date:** [October 22, 2023, 9:34pm UTC](https://discourse.julialang.org/t/roadmap-for-small-binaries/99266/63 "2023-10-22T21:34:54Z")

</div>

> [@Mason](#):
>
> “want statically compiled small binaries? Make your code completely type stable and non-allocating”

That sounds very restrictive. C++ allows you to have type instabilities in a controlled way (runtime dispatch with inheritance and virtual functions), allocations with RAII, and exceptions (also not supported by StaticCompiler.jl).

> [@mkitti](#):
>
> dynamically dispatch which might require the Julia runtime

How large is the part of the runtime dedicated to dynamic dispatch? (Assuming LLVM is not needed as all methods have been precompiled.) Some languages with dynamic dispatch are implemented with a tiny amount of code, e.g. FemtoLisp. Obviously Julia’s multiple dispatch rules are more complex, but is the codebase for disptach really that big?

---

<div class="post-metadata">

**Author:** ![nsajko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nsajko/32/221187_2.png) [@nsajko](https://discourse.julialang.org/u/nsajko)\
**Post date:** [October 22, 2023, 9:38pm UTC](https://discourse.julialang.org/t/roadmap-for-small-binaries/99266/64 "2023-10-22T21:38:04Z")

</div>

> [@joa-quim](#):
>
> Why are those sooo big?

To play devil’s advocate, why does it matter? I mean, really, Julia’s compiled code size doesn’t affect me at all. It seems like there are tons of other more exciting stuff that Julia devs could do (and are doing) than optimizing storage costs of compiled code.

---

<div class="post-metadata">

**Author:** ![joa-quim](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/joa-quim/32/227_2.png) [@joa-quim](https://discourse.julialang.org/u/joa-quim)\
**Post date:** [October 22, 2023, 9:43pm UTC](https://discourse.julialang.org/t/roadmap-for-small-binaries/99266/65 "2023-10-22T21:43:14Z")

</div>

Isn’t this a thread about small binaries?

---

<div class="post-metadata">

**Author:** ![jar1](https://avatars.discourse-cdn.com/v4/letter/j/c0e974/32.png) [@jar1](https://discourse.julialang.org/u/jar1)\
**Post date:** [October 22, 2023, 9:57pm UTC](https://discourse.julialang.org/t/roadmap-for-small-binaries/99266/66 "2023-10-22T21:57:41Z")

</div>

> [@Could Someone Explain To Me the Desire for Small Julia Binaries?](https://discourse.julialang.org/t/could-someone-explain-to-me-the-desire-for-small-julia-binaries/96484):
>
> Hi folks, I’ve seen this notion pop-up repeatedly about wanting small binaries created within the Julia ecosystem. I personally have been quite confused about this desire due to probably not working in a space where having small binaries is desired. My mental oversimplification of this problem is that as personal computer storage space has gotten so large, having big binaries is not a problem any more – as I see it. But I must be missing something crucial. Could anyone explain to me why we wa…

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [October 22, 2023, 10:08pm UTC](https://discourse.julialang.org/t/roadmap-for-small-binaries/99266/67 "2023-10-22T22:08:56Z")

</div>

> [@greatpet](#):
>
> How large is the part of the runtime dedicated to dynamic dispatch? (Assuming LLVM is not needed as all methods have been precompiled.) Some languages with dynamic dispatch are implemented with a tiny amount of code, e.g. FemtoLisp. Obviously Julia’s multiple dispatch rules are more complex, but is the codebase for disptach really that big?

This is the runtime without code generation.

```julia
$ du -hcs ~/.julia/juliaup/julia-1.9.3+0.x86.linux.gnu/lib/julia/libjulia-internal.so.1.9
11M	~/.julia/juliaup/julia-1.9.3+0.x86.linux.gnu/lib/julia/libjulia-internal.so.1.9
11M	total

$ du -hcs ~/.julia/juliaup/julia-1.10.0-beta2+0.x64.linux.gnu/lib/julia/libjulia-internal.so.1.10.0
13M	~/.julia/juliaup/julia-1.10.0-beta2+0.x64.linux.gnu/lib/julia/libjulia-internal.so.1.10.0
13M	total

```

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [October 22, 2023, 10:47pm UTC](https://discourse.julialang.org/t/roadmap-for-small-binaries/99266/68 "2023-10-22T22:47:08Z")

</div>

> [@nsajko](#):
>
> To play devil’s advocate, why does it matter? I mean, really, Julia’s compiled code size doesn’t affect me at all.

And same thing with performance. Do we really care about an extra half a second here and there? Must we always be in such a hurry, when perhaps a leisurely and relaxed program pace is at least as satisfying?

---

<div class="post-metadata">

**Author:** ![anon56330260](https://avatars.discourse-cdn.com/v4/letter/a/f07891/32.png) [@anon56330260](https://discourse.julialang.org/u/anon56330260)\
**Post date:** [October 22, 2023, 10:53pm UTC](https://discourse.julialang.org/t/roadmap-for-small-binaries/99266/69 "2023-10-22T22:53:54Z")

</div>

It doesn’t work like this. For small binaries, you need to eliminate runtime by discarding all the method tables and useless binaries, especially those in Base and stdlib library, as they are rather huge codebases. If you compile with `Number`, then you have to save every potential method tables at runtime (so that you can lookup them), this totally ruins the goal of small binaries.

You may ask “Why there are useless binaries”? The answer is that Julia has generics and specializations. Precompilation saves more binary codes than you may need them. For example, one user uses `f(Vector{Int})` and another uses `f(Vector{Float64})`. A library author can include both `f(Vector{Int})` and `f(Vector{Float64})` in the precompiled codes, but each of two uses need only one copy from the cache.

A common process called “Tree shaking” can be used to discard unused binaries. It works by figuring out what methods are unused (unreachable) and then deletes them. This requires the code to be statically typed, otherwise you quickly lose precison in the call graph and include every method eventually (so tree shaking has no effect at all).

I will emphasize once more: for small binaries there is few room for any fancy PL/Compiler stuff. So for practical consideration, please stop thinking about the possibility of runtime multiple dispatches/union splitting/whole program analysis(tree shaking). Let’s simply stick to type stable code (in a restricted sense).

> [@greatpet](#):
>
> That sounds very restrictive. C++ allows you to have type instabilities in a controlled way (runtime dispatch with inheritance and virtual functions), allocations with RAII, and exceptions (also not supported by StaticCompiler.jl).

You are 100% right. That’s why I said:

> [@anon56330260](#):
>
> There are more essential difficulties than simply invoking `Core.Compiler` `LLVM` and linker to assemble binary products.

The adoption of such model is an essential difficulty. A restrictive compilation model is unattractive in any way. So even though I use a equally restrictive model for my TypeChecker.jl, I really hate it and know it must be improved. But I can’t improve it by myself, because it requires modification to core language (like adding OOP), while I don’t have any right to do so.

That’s why you can see a lot of people try to defend the possibility of using multiple dispatch at runtime (which is simply a waste of time in my opinion, at least for the purpose of small binaries), to make the model look less restrictive. Without modification of core language we can only have these bad workarounds, but none of them is satisfying.

---

<div class="post-metadata">

**Author:** ![nsajko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nsajko/32/221187_2.png) [@nsajko](https://discourse.julialang.org/u/nsajko)\
**Post date:** [October 23, 2023, 8:25am UTC](https://discourse.julialang.org/t/roadmap-for-small-binaries/99266/70 "2023-10-23T08:25:47Z")

</div>

> [@anon56330260](#):
>
> it requires modification to core language (like adding OOP)

What do you mean by “adding OOP”? You mean inheritance from concrete types? How would that help with binary size?

---

<div class="post-metadata">

**Author:** ![anon56330260](https://avatars.discourse-cdn.com/v4/letter/a/f07891/32.png) [@anon56330260](https://discourse.julialang.org/u/anon56330260)\
**Post date:** [October 23, 2023, 11:15pm UTC](https://discourse.julialang.org/t/roadmap-for-small-binaries/99266/71 "2023-10-23T23:15:19Z")

</div>

We need alternatives for runtime multiple dispatch because of its non-trivial implementation costs, otherwise we can only have a restrictive static compilation model. So OOP is unavoidable (or at least some kind of function dispatch table/function pointers). If you don’t offer an alternative, people then opt for runtime multiple dispatch, which renders their codes non-compilable.

Reiterate my point again:

> [@anon56330260](#):
>
> There are more essential difficulties than simply invoking `Core.Compiler` `LLVM` and linker to assemble binary products.

Please read all the comments in this thread carefully to examinate why it’s so hard to add small binaries to Julia. It’s much harder than you may have thought to object my argument.

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [October 24, 2023, 12:17am UTC](https://discourse.julialang.org/t/roadmap-for-small-binaries/99266/72 "2023-10-24T00:17:57Z")

</div>

> [@anon56330260](#):
>
> We need alternatives for runtime multiple dispatch

We just need to “close” or “seal” methods at some point so that we have a finite method table to dispatch upon. Perhaps we could even reduce the size of a particular dynamic method dispatch with limited knowledge of the possible types.

---

<div class="post-metadata">

**Author:** ![nsajko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nsajko/32/221187_2.png) [@nsajko](https://discourse.julialang.org/u/nsajko)\
**Post date:** [October 24, 2023, 12:54am UTC](https://discourse.julialang.org/t/roadmap-for-small-binaries/99266/73 "2023-10-24T00:54:03Z")

</div>

> [@anon56330260](#):
>
> We need alternatives for runtime multiple dispatch because of its non-trivial implementation costs, otherwise we can only have a restrictive static compilation model.

I kind of get this point, but I don’t see why you’d want non-abstract inheritance specifically. I think you’d be restricted by the static compilation requirements just the same, even if Julia got non-abstract inheritance. Julia without run-time dispatch is about as powerful as C++ without run-time dispatch, with or without inheritance, I think?

In any case, it’s curious to me that inheritance is still widely used in the industry, considering that it’s long been recognized as leading to bad design (“prefer composition over inheritance”). The “gang of four” Design Patterns book, which used to be somewhat of an OOP bible AFAIK, espouses this view, too.

> [@anon56330260](#):
>
> or at least some kind of function dispatch table/function pointers

Yeah, some kind of function pointer type (which would _not_ be parameterized by the function type, to facilitate type-stable access to collections of function pointers) would clearly be useful for code that has to be statically compiled.

---

<div class="post-metadata">

**Author:** ![anon56330260](https://avatars.discourse-cdn.com/v4/letter/a/f07891/32.png) [@anon56330260](https://discourse.julialang.org/u/anon56330260)\
**Post date:** [October 24, 2023, 2:37am UTC](https://discourse.julialang.org/t/roadmap-for-small-binaries/99266/74 "2023-10-24T02:37:15Z")

</div>

> [@mkitti](#):
>
> We just need to “close” or “seal” methods at some point so that we have a finite method table to dispatch upon. Perhaps we could even reduce the size of a particular dynamic method dispatch with limited knowledge of the possible types.

You can firstly try this on the Base library. I don’t buy it unless you have successfully reproduce a PoC of “tree shaking with limited knowledge of the possible types” on a large project, stably. I have said that you are wasting your time on considering all these (either broken or overly restrictive) variants of runtime multiple dispatch.

Firstly, Julia’s subtyping type system is rather complicated and actively abused in practice. (Any/bounds/where/Vararg/NamedTuple). Even simple facts about this system, like intersection of two types become too hard to calculate. I really double your limited knowledge can yield anything useful, as you can lose precision in some commonly-used methods like `iterate` and `+`. Secondly, this is still whole program analysis. I have said many times in this thread it’s impractical to do this. Also, I really can’t see the difference between “closed method” table and union splitting. As long as you have abstract type, it loses precison quite quickly.

> [@nsajko](#):
>
> Julia without run-time dispatch is about as powerful as C++ without run-time dispatch, with or without inheritance, I think?

Do Julia users really want to use “Julia/C++ without run-time dispatch”?

> [@nsajko](#):
>
> In any case, it’s curious to me that inheritance is still widely used in the industry, considering that it’s long been recognized as leading to bad design (“prefer composition over inheritance”). The “gang of four” Design Patterns book, which used to be somewhat of an OOP bible AFAIK, espouses this view, too.

Then you and the OOP bible are wrong. The pattern should be “use inheritance” instead of “use composition”. That’s it. Of course, it’s your freedom to prefer composition over inheritance. But then it’s also other people’s freedom to write “anti-pattern” codes with inheritance. Please calibrate your view with the real world practice instead of opinions on the book.

> [@nsajko](#):
>
> Yeah, some kind of function pointer type (which would _not_ be parameterized by the function type, to facilitate type-stable access to collections of function pointers) would clearly be useful for code that has to be statically compiled.

Much like C, but this doesn’t quite work with Julia’s subtyping system as Julia has no `void*` (unless you use Ptr{Nothing} but this will not look like Julia anymore). You can’t create pointer of `f(x::AbstractType)::y`. You can only create `f(x::ConcreteType)::y`. But without `void*`, pointers of different concrete types will be different unless you want to introduce variance on arrow types.

---

<div class="post-metadata">

**Author:** ![ToucheSir](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/touchesir/32/14411_2.png) [@ToucheSir](https://discourse.julialang.org/u/ToucheSir)\
**Post date:** [October 24, 2023, 3:28am UTC](https://discourse.julialang.org/t/roadmap-for-small-binaries/99266/75 "2023-10-24T03:28:11Z")

</div>

> [@anon56330260](#):
>
> Much like C, but this doesn’t quite work with Julia’s subtyping system as Julia has no `void*` (unless you use Ptr{Nothing} but this will not look like Julia anymore). You can’t create pointer of `f(x::AbstractType)::y`. You can only create `f(x::ConcreteType)::y`. But without `void*`, pointers of different concrete types will be different unless you want to introduce variance on arrow types.

Without commenting on the rest of the points, I thought `Any` (or `jl_value_t*` as represented behind the scenes) fills the niche of a `void*` equivalent? This is based on it being possible to call [Julia functions from C](https://docs.julialang.org/en/v1/manual/embedding/#Calling-Julia-Functions) if all arguments are boxed to `jl_value_t*` beforehand, and the existence of a formal (though not well documented) calling convention in [Calling Conventions · The Julia Language](https://docs.julialang.org/en/v1/devdocs/callconv/). I agree that this doesn’t really solve the problem the way people are looking for because it degrades all non-static dispatches to the OO language equivalent of `Object ret = func(Object[] args...)`, nor does it solve the tree-shaking problem.

---

<div class="post-metadata">

**Author:** ![anon56330260](https://avatars.discourse-cdn.com/v4/letter/a/f07891/32.png) [@anon56330260](https://discourse.julialang.org/u/anon56330260)\
**Post date:** [October 24, 2023, 4:33am UTC](https://discourse.julialang.org/t/roadmap-for-small-binaries/99266/76 "2023-10-24T04:33:18Z")

</div>

You are right. Any == jl\_value\_t\* == void\* but it’s an implmentation thing. Any is formally treated on a type level (in a type inferencer). Its layout is unknown (not necessarily a pointer). Type inferencer has freedom to process it differently. Consider this function:

```julia
x = Ref{Any}(1)
push!(Vector{Any}, x[])

```

Is this type stable? We can implement this with no dynamic method call, as long as we treat Any as a void pointer (and by inlining push!). But Julia inferencer **may** decide that it should specialize the `push!` call and generate a `jl_apply_generic` of `push!` instead. As long as we use the builtin type inferencer to generate typed IR, using Any is unreliable and we must use `Ptr{Nothing}` to explicitly express the fact that we want a boxed generic pointer.

> [@ToucheSir](#):
>
> I agree that this doesn’t really solve the problem the way people are looking for because it degrades all non-static dispatches to the OO language equivalent of `Object ret = func(Object[] args...)`, nor does it solve the tree-shaking problem.

Yeah. These are basically problems of Julia’s type systems (compiler frontend). I don’t really think we can magically solve them without significantly changing Julia’s semantics. One thing I have painfully learned in my prototype implementation is that we must do the right thing at the right place. It’s just not static compiler and linker’s responsibility to patch or solve these problems. Why? Because it’s super hard to debug and correct them at this stage, just think about those unreadable linker error. No one will try to maintain such a complicated and highly-coupled implementation.

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [October 24, 2023, 7:02am UTC](https://discourse.julialang.org/t/roadmap-for-small-binaries/99266/77 "2023-10-24T07:02:33Z")

</div>

> [@anon56330260](#):
>
> Do Julia users really want to use “Julia/C++ without run-time dispatch”?

For me, absolutely.

I would be fascinated to hear the other side of this discussion. Instead of ‘how difficult is it to compile unstable code’, the question is ‘how difficult will it be to compile completely type stable code, with no dynamic dispatch/union splitting/`Any`/abstract types etc?’ How much of the language could reasonably be supported by a static compiler in the not-too-distant future? (By ‘type stable’ I mean whatever the compiler is able to figure out statically).

(If this has been discussed further up, I would appreciate a link 🙂 )

---

<div class="post-metadata">

**Author:** ![adienes](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/adienes/32/37459_2.png) [@adienes](https://discourse.julialang.org/u/adienes)\
**Post date:** [October 24, 2023, 12:34pm UTC](https://discourse.julialang.org/t/roadmap-for-small-binaries/99266/78 "2023-10-24T12:34:13Z")

</div>

> [@DNF](#):
>
> How much of the language could reasonably be supported by a static compiler in the not-too-distant future?

pretty sure I could use almost my entire project with just `Float64` and `Vector{Float64}` 👀

---

<div class="post-metadata">

**Author:** ![jar1](https://avatars.discourse-cdn.com/v4/letter/j/c0e974/32.png) [@jar1](https://discourse.julialang.org/u/jar1)\
**Post date:** [October 24, 2023, 5:39pm UTC](https://discourse.julialang.org/t/roadmap-for-small-binaries/99266/79 "2023-10-24T17:39:12Z")

</div>

How do you do error paths in your project?

---

<div class="post-metadata">

**Author:** ![adienes](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/adienes/32/37459_2.png) [@adienes](https://discourse.julialang.org/u/adienes)\
**Post date:** [October 24, 2023, 5:45pm UTC](https://discourse.julialang.org/t/roadmap-for-small-binaries/99266/80 "2023-10-24T17:45:57Z")

</div>

what do you mean?

[Previous page](https://discourse.julialang.org/t/roadmap-for-small-binaries/99266.md?page=3)

[Next page](https://discourse.julialang.org/t/roadmap-for-small-binaries/99266.md?page=5)