Pushing the Limits of Small Binary Creation

General Usage
,
1

Hi all, quick story:

It was at a hotel bar the other day that @asinghvi17 , @avik , @mortenpi , Professor Rebecca Killick, and I did an impromptu hackathon for about 5 hours to see what could be possible with the newly landed PR add --trim option for generating smaller binaries by JeffBezanson · Pull Request #55047 · JuliaLang/julia · GitHub about making small binaries in Julia 1.12. Here’s a write-up of some of our findings and experimentation:

juliac Is Really Great! The juliac compiler that is in development is such a great idea! I highly recommend checking out the blog post by @jbytecode here: juliac on a starters guide for getting started with juliac usage! What we uniquely did here was contribute with showing how to make ccallable functions work within juliac.

Language Interoperability Is Possible with Small Binaries. After much tinkering and exploring, we figured out how to make a small binary that could be executed from within C++ (I’ll let Morten comment more on that!) which evaluated as expected. But what was quite exciting is that after a bunch more tinkering, we got the small binaries to be callable (and usable) from within Julia and R languages!

The Future of juliac Is Exciting! I would say right now, it is still a bit tricky to write juliac compatible code (i.e. type stability, how to use the Julia C/C++ interfaces, write the interface to these binaries from different languages) but there is a lot of richness here. I think we are now at a new frontier for a deeper level of language interoperability that will see Julia proliferate across other ecosystems. I’ll let others comment here!

If you want to reproduce my results and experiments, I made a toy repo here for playing around further!

But, here are some teaser photos showing this in action in Julia:

image
image972×305 26.4 KB

And then in R:

image
image972×789 31.2 KB

There is so much to be done here! Alas, I am going to be busy for the next week or so so probably won’t be able to get back to this soon, but I wanted to write up my thoughts and findings here before I am sucked into other research and coursework endeavors again.

Cheers and please let me know your thoughts and ideas!

~ tcp :deciduous_tree:

45 Likes
2

nice. can’t wait.

1 Like
3

Maybe it’s too early to ask, but what’s a good workflow, when using juliac, to link against a shared library from a JLL package and deploy the compiled binary with proper environments (e.g., LD_LIBARARY_PATH) for the dependencies?

In a complicated projedct with juliac, will you need a build system like Make or CMake? Or will it be more like a usual Julia package workflow with Project.toml and Manifest.toml?

2 Likes
4

Tagging @droodman who might be interested in this to the extent it simplified using Julia as a Stata/R “backend”

5 Likes
5
  1. Can we have an example that requires the runtime? add_julia() from simple.jl is too simple, and would work even with StaticCompiler.jl.
  2. I agree that CMake would make it easier to include in larger (e.g. C++) projects. See the last paragraph here for the Common Lisp version.
2 Likes
6

We were working on kernal smoothing and gibbs examples but the bugs we were getting made us go even simpler to start with. I’m sure we can add an update once we can get back to those.

5 Likes