# Should we distribute multiple official system images for Julia 1.11?

**URL:** <https://discourse.julialang.org/t/should-we-distribute-multiple-official-system-images-for-julia-1-11/107353>\
**Category:** Internals & Design\
**Tags:** repl, pkg, system-image\
**Created:** [December 9, 2023, 10:43am UTC](https://discourse.julialang.org/t/should-we-distribute-multiple-official-system-images-for-julia-1-11/107353 "2023-12-09T10:43:24Z")\
**Posts on this page:** 11\
**Page:** 3

<div class="post-metadata">

**Author:** ![TheLateKronos](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/thelatekronos/32/12824_2.png) [@TheLateKronos](https://discourse.julialang.org/u/TheLateKronos)\
**Post date:** [December 10, 2023, 2:05pm UTC](https://discourse.julialang.org/t/should-we-distribute-multiple-official-system-images-for-julia-1-11/107353/41 "2023-12-10T14:05:33Z")

</div>

The added complexity from different downloadables seems much worse to me than a 400 ms slowdown.

---

<div class="post-metadata">

**Author:** ![juliohm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juliohm/32/215266_2.png) [@juliohm](https://discourse.julialang.org/u/juliohm)\
**Post date:** [December 10, 2023, 3:55pm UTC](https://discourse.julialang.org/t/should-we-distribute-multiple-official-system-images-for-julia-1-11/107353/42 "2023-12-10T15:55:11Z")

</div>

Sharing a small comment here about the interaction between `Pkg` and `Distributed`.

I am used to distributing parallel jobs with a “preamble” code that uses `Pkg` to activate the environment of the script on all worker nodes. The preamble assumes that both `Pkg` and `Distributed` are readily available in any Julia installation on a cluster of computers:

> **[GitHub - Arpeggeo/julia-distributed-computing: The ultimate guide to...](https://github.com/Arpeggeo/julia-distributed-computing)**
>
> The ultimate guide to distributed computing in Julia - GitHub - Arpeggeo/julia-distributed-computing: The ultimate guide to distributed computing in Julia

It would be nice to keep this use case in mind moving forward. Doing distributed computing in Julia is a bit difficult already, and I can imagine that moving `Pkg` outside the system image can affect this usage somehow. Am I wrong?

This is slightly off-topic, but it would be nice to know how these changes are aligned with the future of `Distributed`, `Dagger`, etc.

---

<div class="post-metadata">

**Author:** ![\_bernhard](https://avatars.discourse-cdn.com/v4/letter/_/bc79bd/32.png) [@\_bernhard](https://discourse.julialang.org/u/_bernhard)\
**Post date:** [December 10, 2023, 6:02pm UTC](https://discourse.julialang.org/t/should-we-distribute-multiple-official-system-images-for-julia-1-11/107353/43 "2023-12-10T18:02:05Z")

</div>

> [@Palli](#):
>
> That’s a misunderstanding, you can still use Pkg and REPL with the smaller sysimage, just with a bit higher latency. The alternative sysimage more like the current one.

The only change compared to now would be a slight increase in latency.

---

<div class="post-metadata">

**Author:** ![JM\_Beckers](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jm_beckers/32/22482_2.png) [@JM\_Beckers](https://discourse.julialang.org/u/JM_Beckers)\
**Post date:** [December 10, 2023, 8:05pm UTC](https://discourse.julialang.org/t/should-we-distribute-multiple-official-system-images-for-julia-1-11/107353/44 "2023-12-10T20:05:08Z")

</div>

Please don’t. Do not forget the large majority of Julia users who want to use Julia for their problems, not to develop Julia itself (and are only rarely active here).  
For users, a small latency when adding packages is certainly not a problem whereas adding complexity in the setup of Julia would be.

---

<div class="post-metadata">

**Author:** ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)\
**Post date:** [December 10, 2023, 9:41pm UTC](https://discourse.julialang.org/t/should-we-distribute-multiple-official-system-images-for-julia-1-11/107353/45 "2023-12-10T21:41:49Z")

</div>

I don’t see why there would be multiple downloads or even any need for a user to explicitly choose or be aware of different sysimgs. The `julia` binary already has the ability to load a user-specified sysimg and simply defaults to the standard one. We could ship two different sysimgs—a non-interactive one and an interactive one—and load the former when running a script and the latter when starting an interactive julia session. That said, it would, as Valentin pointed out, prevent upgrading Pkg and REPL without rebuilding the sysimgs.

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [December 11, 2023, 12:54am UTC](https://discourse.julialang.org/t/should-we-distribute-multiple-official-system-images-for-julia-1-11/107353/46 "2023-12-11T00:54:13Z")

</div>

> [@JM\_Beckers](#):
>
> Do not forget the large majority of Julia users who want to use Julia for their problems, not to develop Julia itself (and are only rarely active here).

Yes, I completely understand that, but this is also a source of my concern. Splitting Pkg and REPL out of the system image makes these packages easier to develop. As I have been working on these packages, the workflow is much easier than in the past. They are just normal packages now after all. Does it make Julia easier to use for the large majority of users? My sense here is the primary beneficiaries are advanced users and developers of Julia rather than the majority of users.

The problem is it depends how you use Julia. If you primarily use Julia by starting the REPL, then the changes as committed right now add some additional latency to that startup every single time. For these users, the experience will be less smooth than it is now. We did come up with some mitigations above such as asynchronous loading of Pkg.jl at Julia or REPL startup. This was tremendously useful to me as this prompted me to refactor the new Pkg REPLExt to make it compatible with asynchronous loading.

> [@juliohm](#):
>
> I am used to distributing parallel jobs with a “preamble” code that uses `Pkg` to activate the environment of the script on all worker nodes. The preamble assumes that both `Pkg` and `Distributed` are readily available in any Julia installation on a cluster of computers

For Julio, the packages will load as they did before, but each of these jobs will now take an additional 500 ms to load than they did before. I think Julio is a sophisticated user so I could advise him to build a custom system image including Pkg.jl, but I’m not sure I’m going to be able to do the same for all my colleagues. Across many distributed jobs and runs this additional can add up.

Also if the distributed workflow were running on a serverless platform such as AWS Lambda, that can easily add up to a significant cost. At my institution, these kind of delays forced a rewrite of a significant amount of Java code to a language with less latency.

If your primary concern is about the size of the system image, then the modular Pkg.jl and REPL.jl changes will be of great help to you. The system image will start smaller. It will also be more modular than it is now. Pkg.jl will no longer directly depend on the REPL as a strong dependency.

My sense is that many new users to Julia will start with the REPL. I did. Proceeding as it is now, their Julia 1.11 experience will be less smooth than the Julia 1.10 experience. What I would like is the best of both worlds, 1) a fast, responsive REPL and package manager, and 2) a small deployment profile. This is possible, but perhaps not with a single system image A monolithic system image with everything I need will be faster than a modular one. We can hopefully narrow the difference, but there will be a difference. At the end of the day, a modular Julia gives us options, be we need to exercise them.

Recall from the Julia survey that the second highest complaint is TTFX [1]. A half second additional latency is very noticeable in interactive usage. While not terrible, I do not think we should so easily brush off this regression.

As Stephan outlined, having multiple system images is possible and can be completely transparent to the end user. The changed core Julia workflow is as follows.

1. Are we running interactively? (This step already exists)
2. If yes, we are running interactively, check if “.julia/sysimages/isys.so” exists at the pre-defined location and load it by default. (This step partially exists due the -J option)
3. If we are not running interactively or if “isys.so” does not exist, load the normal “sys.so”.

Outside of core Julia, we then need a package or two to manage downloading registered system images as artifacts and either symlinking them into the designated location or copying them there. Maybe it would be better to have a config file, but that is an implementation detail. When someone posts, “Why is Julia so laggy?”, we can respond “`] add REPLSystemImage`” instead of “Step 1: Build a list of cryptic precompile statements or assemble a comprehensive workflow script…”. Maybe later there will be a `SystemImageManager` or we can update [`JuliaManager.jl`](https://github.com/tkf/JuliaManager.jl).

Anyways, I thought it would be good to start thinking about this early in the 1.11 release cycle than later. I’m willing to work on it.

[1] [Julia User & Developer Survey 2023](https://info.juliahub.com/julia-user-developer-survey-2023)

---

<div class="post-metadata">

**Author:** ![nsslh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nsslh/32/27447_2.png) [@nsslh](https://discourse.julialang.org/u/nsslh)\
**Post date:** [December 11, 2023, 6:42am UTC](https://discourse.julialang.org/t/should-we-distribute-multiple-official-system-images-for-julia-1-11/107353/47 "2023-12-11T06:42:10Z")

</div>

Is there a binary thinking trap, here?

Is it possible for REPL and Pkg to be separate packages, that just happen to be bundled into the sysimages and yet can still be upgraded independently? Or was the goal of extracting them really to produce a smaller sysimage?

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [December 11, 2023, 6:46am UTC](https://discourse.julialang.org/t/should-we-distribute-multiple-official-system-images-for-julia-1-11/107353/48 "2023-12-11T06:46:44Z")

</div>

> [@nsslh](#):
>
> Is it possible for REPL and Pkg to be separate packages, that just happen to be bundled into the sysimages and yet can still be upgraded independently?

No. In this case, to upgrade the packages, one would have to compile a new system image. Although, if the system image were provided by a package, then perhaps the system image could be upgraded by upgrading the package.

> [@nsslh](#):
>
> Or was the goal of extracting them really to produce a smaller sysimage?

Yes, in part, among the other benefits.

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [December 11, 2023, 8:13am UTC](https://discourse.julialang.org/t/should-we-distribute-multiple-official-system-images-for-julia-1-11/107353/49 "2023-12-11T08:13:29Z")

</div>

I wrote a prototype for system image switching. In the prototype, we attempt to load `isys.so` rather than `sys.so` if we are running interactively. If `isys.so` cannot be loaded, we fallback to use `sys.so`.

> <https://github.com/JuliaLang/julia/pull/52482>
>
> If Julia is running interactively, attempt to load \`isys.so\` rather than \`sys.so…\` as the system image. If \`isys.so\` is not found, fallback to using \`sys.so\`. This allows the user or perhaps a package to install a distinct system image for interactive use.
> 
> This pull request arose as a proof of concept from a \[Discourse discussion\](https://discourse.julialang.org/t/should-we-distribute-multiple-official-system-images-for-julia-1-11/107353).
> 
> The idea is that the user may want a system image that includes REPL.jl, Pkg.jl, and other interactive packages (e.g. OhMyREPL.jl) when using Julia interactively due to faster load times. However, having these in the system image is not necessary for non-interactive use. The REPL.jl and Pkg.jl packages were removed from the standard system image in Julia 1.11.
> 
> The following new C API functions are added.
> \* \`jl\_is\_interactive\`
> \* \`jl\_get\_interactive\_sysimg\_path\`
> \* \`jl\_preload\_successful\`
> 
> \# Demonstration
> 
> Here's a demonstration of how this currently works.
> 
> Initially, \`isys.so\` does not exist, so Julia will load \`sys.so\`.
> 
> \`\`\`
> ~/src/julia$ ./julia --banner=no
> julia\> sysimg = Base.JLOptions().image\_file |\> unsafe\_string
> "/home/mkitti/src/julia/usr/lib/julia/sys.so"
> 
> julia\> cp(sysimg, replace(sysimg, "sys.so" =\> "isys.so"))
> "/home/mkitti/src/julia/usr/lib/julia/isys.so"
> 
> julia\> exit()
> \`\`\`
> 
> We copied sys.so to isys.so. Now \`isys.so\` exists, so we shoud load it in interactive mode.
> 
> \`\`\`
> ~/src/julia$ ./julia --banner=no
> julia\> sysimg = Base.JLOptions().image\_file |\> unsafe\_string
> "/home/mkitti/src/julia/usr/lib/julia/isys.so"
> \`\`\`
> 
> If we are not running in interactive mode, then the regular \`sys.so\` is loaded despite \`isys.so\` existing.
> \`\`\`
> ~/src/julia$ ./julia -e "Base.JLOptions().image\_file |\> unsafe\_string |\> println"
> /home/mkitti/src/julia/usr/lib/julia/sys.so
> \`\`\`
> 
> With these changes, the user can create an isys.so that is optimized for interactive use while keeping a separate sys.so for non-interactive use.
> 
> \# Questions
> 
> 1. Where should the interactive system image be located? If this is meant to be controlled by the user, perhaps \`~/.julia/sysimgs/\` would be a good alternative place.
> 2. Should we use \`jl\_is\_interactive\` to set the global variable \`Base.is\_interactive\` or implement \`Base.isinteractive()\`?

Here’s a demonstration of how this currently works.

Initially, `isys.so` does not exist, so Julia will load `sys.so`.

```julia
~/src/julia$ ./julia --banner=no
julia> sysimg = Base.JLOptions().image_file |> unsafe_string
"/home/mkitti/src/julia/usr/lib/julia/sys.so"

julia> cp(sysimg, replace(sysimg, "sys.so" => "isys.so"))
"/home/mkitti/src/julia/usr/lib/julia/isys.so"

julia> exit()

```

We copied sys.so to isys.so. Now `isys.so` exists, so we should load it in interactive mode.

```julia
~/src/julia$ ./julia --banner=no
julia> sysimg = Base.JLOptions().image_file |> unsafe_string
"/home/mkitti/src/julia/usr/lib/julia/isys.so"

```

If we are not running in interactive mode, then the regular `sys.so` is loaded despite `isys.so` existing.

```julia
~/src/julia$ ./julia -e "Base.JLOptions().image_file |> unsafe_string |> println"
/home/mkitti/src/julia/usr/lib/julia/sys.so

```

One complication is Julia’s interactive status is partially determined in Julia after the system image is loaded.

> <https://github.com/JuliaLang/julia/blob/2fe4190b3d26b4eee52b2b1b1054ddd6e38a941e/base/client.jl#L271-L272>

If you run stock `julia` with no arguments, the `Base.JLOptions().isinteractive` will be false while `isinteractive()` will be true.

```julia
julia> Base.JLOptions().isinteractive
0

julia> isinteractive()
true

```

Julia is interactive if the following conditions are met.

1. The `-i` or `--interactive` is provided at the command line OR
2. No `-e` or `-E` commands are provided AND no non-switch arguments are provided AND STDIN is a TTY.

To alleviate this, I wrote a C routine `jl_is_interactive`

---

<div class="post-metadata">

**Author:** ![SteffenPL](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/steffenpl/32/206270_2.png) [@SteffenPL](https://discourse.julialang.org/u/SteffenPL)\
**Post date:** [December 11, 2023, 8:42am UTC](https://discourse.julialang.org/t/should-we-distribute-multiple-official-system-images-for-julia-1-11/107353/50 "2023-12-11T08:42:07Z")

</div>

Would Pkg.jl and REPL.jl be pinned by default if users have the corresponding system image? I guess if we get into a “two Julia flavors” stage, it looks like the common source for errors…

I guess we all want a short `time-to-first-status`. However, it could be _educational_ for a broad base of users if Pkg.jl and REPL.jl are not included per default, but instead, one provides very good tools to quickly add them. The tools to deal with sysimages might gain more wide-spread use over time which might be great for many people who are currently unaware of sysimages.

---

<div class="post-metadata">

**Author:** ![juliohm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juliohm/32/215266_2.png) [@juliohm](https://discourse.julialang.org/u/juliohm)\
**Post date:** [December 11, 2023, 9:45am UTC](https://discourse.julialang.org/t/should-we-distribute-multiple-official-system-images-for-julia-1-11/107353/51 "2023-12-11T09:45:02Z")

</div>

Thank you Mark for explaining the situation with Distributed jobs and the expected additional delay per process. That is something to consider carefully in the context of parallel cloud services.

If the binary is split into two, it would be great if the official installation instructions had a section dedicated to HPC clusters written for sysadmin people who never heard of Julia.

Back in the days when I was working at IBM I had to request the installation of the language at a HPC cluster and the person responsible for the installation didn’t know where to start. I can imagine that this split will worsen sysadmin lives as well. We need official guidance on how to install Julia in HPC clusters, clouds, … besides the single-user, single-process installation.

I also believe that this change could be bundled into Julia v2.0 to reduce the impact on the user base, which is hopefully increasing with the v1.x series of improvements.

[Previous page](https://discourse.julialang.org/t/should-we-distribute-multiple-official-system-images-for-julia-1-11/107353.md?page=2)