Julia installation file sizes

General Usage
1

Hi,

Julia install sizes has been steadily growing in size. It was on the range 400-600 MB for versions < 1.10 (can’t remember anymore) and is.
v1.10 ~700 MB
v1.11 ~870 MB
v112 ~0.99 GB (nightly)

I found that the main responsible for this size increases is the contents of the ...share\julia\compiled\v1.XX directory, that for nightly has ~437 MB (~44 MB for Julia 1.10).

But the weirdest of all is that all shared libs in those directories are duplicated. For example:

C:\programs\Julia-1.12\share\julia\compiled\v1.12\Pkg>dir
 Volume in drive C is OS
 Volume Serial Number is B071-46FE

 Directory of C:\programs\Julia-1.12\share\julia\compiled\v1.12\Pkg

08/21/2024  05:19 PM    <DIR>          .
08/21/2024  05:19 PM    <DIR>          ..
08/18/2024  05:05 AM        85,742,592 tUTdb_e6ieL.dll
08/21/2024  05:33 PM           599,714 tUTdb_e6ieL.ji
08/18/2024  05:05 AM        94,069,248 tUTdb_Miz3B.dll
08/21/2024  06:01 PM           599,714 tUTdb_Miz3B.ji

Is this intended to be like that or some environment duplication during Julia building is making it this way?

3 Likes
2

I believe during the build stdlibs are precompiled with different configurations, which results in multiple (which is different from duplicate, which suggests they are the same thing) libraries.

3

Yes, duplication was not a good word, sorry, but the question remains. Why do we want multiple configurations and hence file size ~duplication?

4

To avoid having to wait to compile stdlibs in different cases.

1 Like
5

I believe the specific configurations are the ones for regular running and the ones for testing (testing has code coverage and bounds-check=true iirc)

4 Likes
6

Testing as for when we run package tests for example?

1 Like
7

yes.

8

If you want to delete either/all except main, is there a way to find out which is which?

9

Coming back to this. I made this experiment with Julia nightly. Since I noticed that when running tests a second precompile cache dll is generated and is bigger than the regular one, I went to ...\Julia-1.12\share\julia\compiled\v1.12, looked for the name termination of the larger .dll (sorry, did not take note of it) and deleted all of those dlls. That shaved 189 MB in that directory. Next run the GMT tests and all went fine. So, again, are those double dlls really needed?

2 Likes
10

Julia installation size is increasing after every version, I was wondering if there is a plan to reduce it? Or at least give the user the means to do it?

11

It seems that during the build process stdlibs are compiled with the 2 differents configurations:
1/ -g2 -O3
2/ -g2 -O3 --check-bounds=yes

Before that it used to be -O2 and -O2 --check-bounds=yes which makes more sense to me because this is the default options used when you start Julia and when you run tests.

If the goal is to avoid having to wait why has this change been made? Am I missing something? I believe @jameson made that change, no?

12

I believe it’s -O3 since it’s faster (and deemed safe), but yes will be larger. And starting with -O2 (default, or lower like -O1), will not recompile. But starting with -O3 (before) would have recompiled stdlib to -O3 to match.

You can compile your own Julia/sysimage with -O2 or lower.

13

Just noticed that Julia nightly size stands now at 1.5G (at least on windows)…

2 Likes
14

Removed ~450 MB of static libs in ...\Julia-1.12\lib\julia\ (sysbase-o.a, basecompiler-o.a), plus 200 MB of *.dll in ...\Julia-1.12\share\julia\compiled\v1.13. A total of ~650 MB and still see no side effects.

15

the side effect would be higher latency under certain configurations.
How did you measure that?

1 Like
16

I don’t know any way of measuring that difference in latency.

But today, just in a slightly related mater, a student dint want to install Julia in her laptop because she hadn’t much more space left. In those cases 600 MB make a difference.

17

there’s definitely a case for a light version…And doubling the size for testing packages with checkounds=yes, seems like something that could be served by a dev version.
But with all of these things, someone needs to care and put time into it.

This is in polarizing trade off land, where half of the community will complain one way or the other, so demos and detailed data for the trade offs is the only way to convince here I suppose!

I do think though, that theres a real regression for the size of the precompiled binaries itself, which I hope is on the radar for improving before a release.

1 Like
18

When you don’t have these numbers, even windows becomes really slow. You don’t want to write code on such machine, even for a student, they can’t follow in class. I’ve had that problem.

Nowadays most, even older, laptops (for ppl with very limited budgets) come with ample space. The student should delete something else.

19

I understand that the dlls built with checkounds=yes might be needed in some situations. What I don’t understand is why there are no errors when I run tests that, by themselves, create precompiled dlls that are also larger because of that default checkounds=yes

But there is more, nightly now ships a static lib, ...\lib\julia\sysbase-o.a that has 358 MB and if I remove it there seems to be no consequence. v.11 does not ship that static lib and I think neither did previous nightlies (how far back, I dont know).

20

There are no errors because if a pkgimage is missing it’s simply recompiled. But that takes time,

1 Like