How close is StaticCompile?

Hello, I am new to the Julia community, and I want to thank you for making a terrific language, wonderful free tools, and very supportive experts that answer questions quickly on Slack and Stack Overflow. I am a big fan.

It was suggested that I raise this issue here

I am a commercial developer who is looking at Julia as a replacement for Python/Cython/C++. I am making an ML product for an employer that is a Java/Python shop. I have written a proto-type in Julia that run 10x faster than my Python version. It is currently being evaluated by several very large customers.

To sell Julia to engineering, I need to be able to package my code in a way that Java and Python can call it, but other engineers do not need to know about Julia. I can not control the architecture, so I need to produce a library.

I used PackageCompiler and PyJulia to plug it into our Python code. This is fine for a prototype, but I think engineering will balk at the 200mb size. Also our build masters don’t know Julia and will have trouble maintaining the build process.

I know that StaticCompiler is under development, and this is a common request

https://news.ycombinator.com/item?id=21695331

How far away is it, and where is static compiling in the priority list? How close are we to being able to type ‘make …’ and have Julia code compiled to an optimized library or executable?

Many thanks

Peter

5 Likes

It is unlikely that in the near future small (<100mb) executables will be makable without an extensive test suite. The reason for this is that if you don’t want to include Julia itself in the library, you need to know that you have compiled every method on every possible combination of types which will appear in the program. To know which functions and types to have in the library will require a test suite that exercises every possible use case. Generating everything isn’t an option since a single 5 argument function would then generate over a million different versions if you have 9 different types defined.

If you follow the link above, Keno seems to think it’s possible, perhaps with some restricted semantics.

But in the context of making a C-callable shared library it seems like the surface area that might get called at runtime should be pretty manageable, right? You’ll presumably need to write a small C wrapper anyways that takes care of some of the julia initialization and has a specific set of function calls it exposes, with known types. I guess there could be issues if you have some dynamic dispatch that might invoke the compiler for some input values and not others, but if the code is pretty type-stable it seems like that should be doable, right?

1 Like

Yeah, that should work. That wrapper is an example of the type of test suite I was talking about.

Was that the only problem? For example, didn’t they complain that libjulia completely swallows SIGINT (and so ctrl-c stops working)? Limitations — PyJulia 0.6.1 documentation

1 Like

I haven’t shown it to them yet. Thanks for the warning :slight_smile: