Yet another question about Julia's high RAM usage

[ParadaCarleton]

@jakobnissen

But aside from the direct quote, it’s also obvious from the design of the language. Julia is an experiment in what happens if you bring everything to the runtime (even a full BLAS with no opt-in!), and do all of the compilation. Specialize on every argument, make every function generic on every argument, make functions and closures their own type. In the generics dilemma, Julia chose the C++ approach and then pushed it to the extreme because C++'s compile times weren’t, apparently, dissuasive enough.

Sounds like this would be a much smaller problem if Julia wasn’t JAOT compiled.

[Mason]

Isn’t it the other way around? The way I see it, the difficulty with this approach is exactly that if you want to fully AOT compile, you have to find, prepare, compile and save every possible reachable specialization.

[Benny]

Yes, AOT compilation needs to cover “everything”, but that isn’t often much compared to a baseline runtime, and there are more factors in what “everything” is. I asked this question in the first place because I didn’t (and still don’t have a way to) know much about how a Julia session allocates source code, docstrings, types, method table, compiled code, standard libraries, the compiler, BLAS buffers, etc. AOT-compiled binaries don’t carry all of that, and it’d be nice to have some idea what is trimmable.

@jakobnissen

Specialize on every argument

There are exceptions to the rule: Function, Type, Vararg arguments needs method parameters, and @nospecialize opts out. But nonspecialized arguments and runtime dispatch have a higher cost and unbounded extent in Julia where functions don’t have consistent return types and are dispatched over multiple arguments; one consequence is trimmed JuliaC being limited to a “static” subset that is less dynamic than many statically typed languages.

[ParadaCarleton]

@Mason

Isn’t it the other way around? The way I see it, the difficulty with this approach is exactly that if you want to fully AOT compile, you have to find, prepare, compile and save every possible reachable specialization.

Fair enough; I was thinking about the TTFP issue rather than binary sizes.

[bensetterholm]

@Oscar_Smith

no one on a personal computer has <8gb of ram.

It’s all fun and games until you try to deploy your “simple” program to a Raspberry Pi and regret your decision of buying the 2 GB model…

[Palli]

@Benny

1. an OpenBLAS implementation that preallocates buffers to accelerate matrix multiplication that cannot be removed from Base

Well, it can be removed from Base, a lot can be removed, I’ve experimented with that:

Faster startup/smaller sysimg, excising Pkg (for now) (#4)

It’s 22% faster startup because the sysimage is 33% smaller.

@bensetterholm

It’s all fun and games until you try to deploy your “simple” program to a Raspberry Pi and regret your decision of buying the 2 GB model…

It helps to make the sysimage smaller. You don’t need to use my fork of Julia, just the sysimage it generates. It’s entirely supported to use Julia with alternative sysimage with -J, --sysimage <file> Start up with the given system image file

Startup of Julia (and memory use) is O(n) in sysimage size, if I recall, roughly. While [Open]BLAS can be dropped, helpful for some users, it only allocates virtual memory, no cost until used.

@ParadaCarleton

Sounds like this would be a much smaller problem if Julia wasn’t JAOT compiled.

And that’s an option now:

@madppiper

Joovy uses three tiers. Tier 0 is basically not-compiled. Fast to load, but slow to execute.

More Python like then. you would also set -O0 before, but then for all code.

[Jake]

@bensetterholm

It’s all fun and games until you try to deploy your “simple” program to a Raspberry Pi and regret your decision of buying the 2 GB model…

I just used top on my 8 GByte RPI 4. The data acquisition takes 1.6 GB (20% CPU) and processing takes 1.9 GB (100 % CPU). Without additional cooling it is probably throttled due to CPU temperature.