# Roadmap for small binaries

**URL:** <https://discourse.julialang.org/t/roadmap-for-small-binaries/99266>\
**Category:** Internals & Design\
**Tags:** question\
**Created:** [May 23, 2023, 8:41am UTC](https://discourse.julialang.org/t/roadmap-for-small-binaries/99266 "2023-05-23T08:41:09Z")\
**Posts on this page:** 20\
**Page:** 6

<div class="post-metadata">

**Author:** ![ToucheSir](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/touchesir/32/14411_2.png) [@ToucheSir](https://discourse.julialang.org/u/ToucheSir)\
**Post date:** [October 31, 2023, 8:18pm UTC](https://discourse.julialang.org/t/roadmap-for-small-binaries/99266/101 "2023-10-31T20:18:38Z")

</div>

> [@thautwarm](#):
>
> Basically, (single) dynamic dispatch itself can be statically compiled with “virtual tables” in .NET CLR/JVM, which can be theoretically much slower than Julia’s multiple dispatch but do not suffer at all from re-compilation and runtime compilation latency.

It just occurred to me that vtables make it trival to completely eliminate a class from compiled code if that class isn’t used anywhere. Thanks for helping me connect the dots!

> [@thautwarm](#):
>
> Dylan’s handling does not really work for Julia due to its lack of support for parametric polymorphism (could be roughly regarded as “generics” tho not precise). Parametric polymorphism is fully supported in Julia.

I’m curious whether this is a fundamental limitation of having parametric polymorphism and multiple dispatch in the same language. To avoid cluttering this thread, created [Reconciling dynamic multimethod dispatch, parametric types and static compilation](https://discourse.julialang.org/t/reconciling-dynamic-multimethod-dispatch-parametric-types-and-static-compilation/105631) if anyone has thoughts or insights.

> [@thautwarm](#):
>
> To solve the issues, the company I’m working for is now concentrating on generating small binaries (both standalone executables and dynamic libraries) for Julia codebase that is “reasonably static”. The project has made progress during mid-2023 and might be available for public use in the early 2024. Our work is based on `code_typed` in a frozen [world age](https://discourse.julialang.org/t/world-age-problem-explanation/9714).

Do you anticipate creating a small writeup to describe how this project works once it’s ready? Sounds exciting!

---

<div class="post-metadata">

**Author:** ![thautwarm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/thautwarm/32/37760_2.png) [@thautwarm](https://discourse.julialang.org/u/thautwarm)\
**Post date:** [November 2, 2023, 7:24am UTC](https://discourse.julialang.org/t/roadmap-for-small-binaries/99266/102 "2023-11-02T07:24:14Z")

</div>

Definitely there will be an article explaining a working AOT compiler.

---

<div class="post-metadata">

**Author:** ![Tarny\_GG\_Channie](https://avatars.discourse-cdn.com/v4/letter/t/3bc359/32.png) [@Tarny\_GG\_Channie](https://discourse.julialang.org/u/Tarny_GG_Channie)\
**Post date:** [November 2, 2023, 1:16pm UTC](https://discourse.julialang.org/t/roadmap-for-small-binaries/99266/103 "2023-11-02T13:16:41Z")

</div>

One big problem on the roadmap to small binaries is that while statically typed languages’ type inferences are normally rule-based, Julia’s type inference is heuristic-based. This means simply adding a static language as a language where all the code has to be type-inferred might mean that the function breaks every now and again. I want to add this point as something not frequently talked about.

---

<div class="post-metadata">

**Author:** ![adienes](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/adienes/32/37459_2.png) [@adienes](https://discourse.julialang.org/u/adienes)\
**Post date:** [November 2, 2023, 1:23pm UTC](https://discourse.julialang.org/t/roadmap-for-small-binaries/99266/104 "2023-11-02T13:23:49Z")

</div>

heuristics are rules, just lots of em mushed together

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [November 2, 2023, 1:27pm UTC](https://discourse.julialang.org/t/roadmap-for-small-binaries/99266/105 "2023-11-02T13:27:10Z")

</div>

This isn’t really correct. Most static languages have heuristics in their inference (since turing complete subtyping is very common). It’s definitely possible to confuse Java’s sub-typing for example.

---

<div class="post-metadata">

**Author:** ![thautwarm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/thautwarm/32/37760_2.png) [@thautwarm](https://discourse.julialang.org/u/thautwarm)\
**Post date:** [December 18, 2023, 8:10pm UTC](https://discourse.julialang.org/t/roadmap-for-small-binaries/99266/106 "2023-12-18T20:10:13Z")

</div>

I’d like to share something interesting 🙂

 ![image](https://global.discourse-cdn.com/julialang/original/3X/0/3/0399ad37289b08b55e2428181fc8ce53184574ed.png)  
Recently, our AOT compiler (made by my team) has nearly achieved full support for type-stable Julia code, including exception handling, multiple dispatch and arbitrary complex constant data.

Something interesting is that our generated native code greatly outperforms vanilla Julia in terms of try-catch-finally statements, while in general cases our AOT Julia is slightly slower than the vanilla.

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [December 18, 2023, 8:25pm UTC](https://discourse.julialang.org/t/roadmap-for-small-binaries/99266/107 "2023-12-18T20:25:57Z")

</div>

Thank you for sharing.

Could you run `@time main()` twice to see the effect of in memory compilation caching?

I would also be interested in seeing the difference in the LLVM IR ot native code generation.

---

<div class="post-metadata">

**Author:** ![thautwarm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/thautwarm/32/37760_2.png) [@thautwarm](https://discourse.julialang.org/u/thautwarm)\
**Post date:** [December 18, 2023, 9:24pm UTC](https://discourse.julialang.org/t/roadmap-for-small-binaries/99266/108 "2023-12-18T21:24:00Z")

</div>

> [@mkitti](#):
>
> Could you run `@time main()` twice to see the effect of in memory compilation caching?

Sure, and I find the cost is stable.

```julia
julia> @time main()
149927
  0.485954 seconds (49.94 k allocations: 780.336 KiB)

julia> @time main()
149861
  0.485176 seconds (49.87 k allocations: 779.305 KiB)

julia> @time main()
149820
  0.484067 seconds (49.83 k allocations: 778.664 KiB)

```

> [@mkitti](#):
>
> I would also be interested in seeing the difference in the LLVM IR ot native code generation.

I’m not sure. In fact, I did nothing special here. Upsilon/Phi-C nodes are directly translated into some C-family language, and the `try-catch` construct in the target language is leveraged to implement stuffs. However, the encoding for Upsilon nodes/PhiC nodes might be quite different from the one that Julia uses:

```C++
block_t _B;

handle_error:

try {
     switch (_block_target(_B)) {
        case 0:
             goto _L1;

        case catch_clause_jump_target1:
             goto _Li;
        ...
        case catch_clause_jump_target2:
            goto _Lk;
     }

    // normal code gen
   
    // Expr(:enter, jump_target_i) =>
       _block_push(_B, jump_target_i);
    
}
catch (exception e) {
    _exception_wrap_and_push(e);

    goto handle_error;
}

```

I don’t have an idea, or I won’t think this interesting. One thought could be our simplification to the stacktrace. So far our AOT Julia creates stacktrace that is more restricted, and more complete stacktrace is available only when the `.dwo/.pdb` files created by the AOT compiler are also kept beside the generated shared libraries/executable.

---

<div class="post-metadata">

**Author:** ![Benny](https://avatars.discourse-cdn.com/v4/letter/b/49beb7/32.png) [@Benny](https://discourse.julialang.org/u/Benny)\
**Post date:** [December 18, 2023, 11:45pm UTC](https://discourse.julialang.org/t/roadmap-for-small-binaries/99266/109 "2023-12-18T23:45:31Z")

</div>

> [@thautwarm](#):
>
> Recently, our AOT compiler (made by my team)…our generated native code greatly outperforms vanilla Julia in terms of try-catch-finally statements, while in general cases our AOT Julia is slightly slower than the vanilla.

This may likely end up in an announcement post anyway if it isn’t closed-source, but can I ask why this compiler diverges from the mainstream compiler rather than repurpose it with additional optimizations assuming no further code is evaluated? Or does it not diverge and it’s slightly slower in general for different reasons?

---

<div class="post-metadata">

**Author:** ![gbaraldi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gbaraldi/32/22101_2.png) [@gbaraldi](https://discourse.julialang.org/u/gbaraldi)\
**Post date:** [December 19, 2023, 2:34pm UTC](https://discourse.julialang.org/t/roadmap-for-small-binaries/99266/110 "2023-12-19T14:34:24Z")

</div>

So while there isn’t a clear roadmap at the moment, it is a goal of the Julia team to have small binaries (It’s the main thing I’ll be working on in the near future). The vision is to have something like `juliac` that takes in a module or a script (this isn’t super clear just yet, but also isn’t super important, it just needs to be something with well defined entry points (main/shared library API) and generates a binary from that.

I believe that from that we will have 3 different options to generate the code, one would be equivalent to what PackageCompiler.jl does right now, the other will be StaticCompiler.jl and the one that’s missing, which will allow using the runtime but will require a well defined call graph (known targets for a possible dynamic dispatch) so that we know what code to compile, and more importantly what to throw away, so that the final binary isn’t too big.

---

<div class="post-metadata">

**Author:** ![AMJ](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/amj/32/214096_2.png) [@AMJ](https://discourse.julialang.org/u/AMJ)\
**Post date:** [December 19, 2023, 5:39pm UTC](https://discourse.julialang.org/t/roadmap-for-small-binaries/99266/111 "2023-12-19T17:39:01Z")

</div>

That’s great news. Thanks for sharing.

As @ChrisRackauckas said in the Julia subreddit, there is also a plan that includes a static subset of Julia. Are there ideas about a little more explicit memory management? For example, we can already use Bumper.jl with StaticCompiler.jl which is a great combination, but there isn’t an interface to use Bumper with most of the language(i.e. Sockets). Or this line of thinking is not in the current plans?

---

<div class="post-metadata">

**Author:** ![gbaraldi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gbaraldi/32/22101_2.png) [@gbaraldi](https://discourse.julialang.org/u/gbaraldi)\
**Post date:** [December 19, 2023, 6:31pm UTC](https://discourse.julialang.org/t/roadmap-for-small-binaries/99266/112 "2023-12-19T18:31:26Z")

</div>

For now the plan is to keep using and improving the GC.

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [December 19, 2023, 7:43pm UTC](https://discourse.julialang.org/t/roadmap-for-small-binaries/99266/113 "2023-12-19T19:43:17Z")

</div>

> [@gbaraldi](#):
>
> the one that’s missing, which will allow using the runtime but will require a well defined call graph (known targets for a possible dynamic dispatch) so that we know what code to compile, and more importantly what to throw away, so that the final binary isn’t too big.

Occasionally, what I want along these lines is just a launcher. A launcher is just an executable that will switch to a specific environment and then run a main function with a full Julia install. On some platforms this is easy enough to do with a shell script or perhaps a small C program. A cross platform way to do this entirely in Julia without depending on a shell or a C compiler would be nice.

---

<div class="post-metadata">

**Author:** ![gitboy16](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gitboy16/32/24906_2.png) [@gitboy16](https://discourse.julialang.org/u/gitboy16)\
**Post date:** [December 19, 2023, 9:14pm UTC](https://discourse.julialang.org/t/roadmap-for-small-binaries/99266/114 "2023-12-19T21:14:36Z")

</div>

You can do that with juliaup (portable version) and a .bat file. I already do it. For linux/apple you’ll need a bash script.  
The bat file download the portable version of juliaup from github, setup some variable in PATH, then juliaup install julia, then launch julia and run some script, then remove everything at the end. That being said not sure what you mean by environment in this context. If it is a julia environment then it is also easy.

(Anyway I think this is off topic)

---

<div class="post-metadata">

**Author:** ![Jose\_Diaz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jose_diaz/32/43750_2.png) [@Jose\_Diaz](https://discourse.julialang.org/u/Jose_Diaz)\
**Post date:** [December 20, 2023, 3:55pm UTC](https://discourse.julialang.org/t/roadmap-for-small-binaries/99266/115 "2023-12-20T15:55:08Z")

</div>

I’m really excited for the AOT compiler

---

<div class="post-metadata">

**Author:** ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)\
**Post date:** [December 21, 2023, 11:26am UTC](https://discourse.julialang.org/t/roadmap-for-small-binaries/99266/116 "2023-12-21T11:26:35Z")

</div>

Yes, that’s a completely different issue though. That’s more a question of defining and better supporting what a Julia app is. We definitely want that as well though. There’s a GitHub issue somewhere.

---

<div class="post-metadata">

**Author:** ![jar1](https://avatars.discourse-cdn.com/v4/letter/j/c0e974/32.png) [@jar1](https://discourse.julialang.org/u/jar1)\
**Post date:** [December 21, 2023, 11:05pm UTC](https://discourse.julialang.org/t/roadmap-for-small-binaries/99266/117 "2023-12-21T23:05:12Z")

</div>

[![](https://global.discourse-cdn.com/julialang/original/3X/4/9/492adc1f1efb23a0a0f30d99f214c29aa0e79627.jpeg "Jeff Bezanson - What's the deal with Julia binary sizes?") ](https://www.youtube.com/watch?v=kNslvU3WD4M)

---

<div class="post-metadata">

**Author:** ![jules](https://avatars.discourse-cdn.com/v4/letter/j/41988e/32.png) [@jules](https://discourse.julialang.org/u/jules)\
**Post date:** [December 22, 2023, 10:48am UTC](https://discourse.julialang.org/t/roadmap-for-small-binaries/99266/118 "2023-12-22T10:48:41Z")

</div>

In the video the example is `x + y`, and while that’s only an example I do feel that math heavy code is more often talked about when we talk about compiling things. To me it seems instead that the bigger “market” with binaries is for string and io heavy code. Because that includes all the general purpose tools like formatters, linters, web servers that you want to deploy somewhere remotely and spin up quickly. Strings, GC and allocations are not at all a focus of GPUCompiler I’d say, and StaticCompiler also doesn’t really allow them for that reason. But I think the community infrastructure would gain much more from work on that than from packaging up purely numeric code. At least I assume that because purely numeric code doesn’t seem like the kind of thing to me where you don’t have the option to run a normal Julia session. Unless targetting embedded devices maybe.  
So what is the optimum we can reach while still including all String, GC and related functionality, just no dynamic compilation?

---

<div class="post-metadata">

**Author:** ![Raf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/raf/32/3383_2.png) [@Raf](https://discourse.julialang.org/u/Raf)\
**Post date:** [December 22, 2023, 11:40am UTC](https://discourse.julialang.org/t/roadmap-for-small-binaries/99266/119 "2023-12-22T11:40:49Z")

</div>

Isn’t the case for purely numeric code replacing C/C++/Fortran in producing binary numeric libraries, e.g. that can be used by R and Python packages, embedded, etc. There’s a huge potential “market” there working directly from julias current strengths.

---

<div class="post-metadata">

**Author:** ![jules](https://avatars.discourse-cdn.com/v4/letter/j/41988e/32.png) [@jules](https://discourse.julialang.org/u/jules)\
**Post date:** [December 22, 2023, 11:57am UTC](https://discourse.julialang.org/t/roadmap-for-small-binaries/99266/120 "2023-12-22T11:57:44Z")

</div>

I’m not a numerics person so my view is heavily skewed of course 🙂 For me, solving the cases where you currently can’t use Julia at all, only because it doesn’t produce small binaries with quick startup time, are more exciting than calling into numerical libraries from python or R, even though it’s also useful to solve that.

[Previous page](https://discourse.julialang.org/t/roadmap-for-small-binaries/99266.md?page=5)

[Next page](https://discourse.julialang.org/t/roadmap-for-small-binaries/99266.md?page=7)
