This month in Julia world - 2024-08

Community Newsletter
1

A monthly newsletter, mostly on julia internals, digestible for casual observers. A biased, incomplete, editorialized list of what I found interesting this month, with contributions from the community.

A bit shorter this time, as the start of the school year is hectic. Many interesting things might have been missed, so do not hesitate to post more underneath.

“Internals” Fora and Core Repos (Slack/Zulip/Discourse/Github):

  • There is significant work on making an ahead-of-time compiler for Julia, capable of producing small static binaries (discussed in the last newsletter and at juliacon). The near-term plan is for that compiler to not support very dynamic Julia code (e.g. Julia code that is not type stable or that looks like python or that relies on compiling new specializations). There are many situations in which such code (and Julia’s capability to work with such code) is extremely valuable, which led to an interesting slack discussion on how the ahead-of-time compiler might lack support for important Julia idioms or how its use might bias future Julia development away from interesting Julia projects that depend on dynamism (e.g. Dagger.jl).
  • The Ordinary Diff Eq ecosystem is one of the jewels of Julia, and naturally it has severely taxed the Julia infrastructure. First it was an example of long import times or times-to-first-solve (not as much as Plots or Makie though), but it was also one of the project spearheadign better precompilation coverage bringing time-to-first-X to zero. But with all this precompilation, it is now one of the slowest packages to install (as compilation happens during install), and OOMs kills are not unheard of. Over the last few months, the SciML team has been carving the diff eq libraries in smaller packages, lowering dependencies and available default solvers, so that users do not compile hundreds upon hundreds of solvers when they need only one. This has now led to very interesting workflow/logistics human scalability questions, leading to a very informative github and slack discussions on how to improve the ergonomics of the “package extension” feature (the one that makes it easy to have code be loaded conditionally on the presence of other packages).
  • “JET, a package which can systematically analyze your code to detect runtime dispatch, now integrates conveniently with Cthulhu, a package for gaining insights about the source of runtime dispatch and other failures of inference.” as Tim announced recently.
  • With the long compilation times we have, is it not annoying when you add a new package and that package decides to update an already installed package, leading to a cascade of new compilations in all other dependent packages. We have a package management configuration that basically forbids that – it is considered to be turned on by default in upcoming versions.
  • It is always fun to discuss the difference between isequal, ==, and === and which one is used for in checks for various containers.

Dustbin of History:

  • Local const is not really a thing in julia, partially because semantics for it are unclear, partially because const checking should be static for performance reasons (a problem not present with global const). Delving deeper into the question of semantics (and the usually unrelated fact that functions are const) leads to some wild edge cases.

In search of contributors and new maintainers (specify novice/moderate/expert and internals/domain background necessary):

  • For a simple way to get into Makie development there is a new set of small changes that the team is making to every single plot recipe, and for which they are eager to onboard new developers. See on github and slack.
  • The Center for Quantum Networks is running many bounties and minigrants on a variety of support packages for quantum information science. Few thousand $ will be paid out already in October for bounties from August.
  • SciML is running a large set of funded small projects – a great opportunity to contribute to the community and be paid for it.
  • ChainRules, a foundational package to the Julia autodiff effort is looking for a new primary maintainer.

Ecosystem Fora, Maintenance, and Colab Promises (Slack/Zulip/Discourse/Github):

  • A ton of new symbolic solvers in Symbolics.jl.
  • A in-depth post on the importance of being able to gradually add Julia-implemented capabilities to a Python codebase. This would benefit the scientific open source ecosystem as a whole and our community in particular, but currently extending Python with C or Rust is easier than extending Python with Julia. On discourse.
  • Moshi.jl, the newest contender for Algebraic Data Types in julia, is receiving a lot of love and in-depth discussion on slack.
  • An interesting discussion on the design and an upcoming breaking release of DifferentiationInterface on slack.
  • A lot of wonderful improvements to ThreadPinning.jl, a library that tells your OS to not move your threads from core to core, helping significantly with cache coherency. On slack.
  • The julia-actions/setup-julia Github CI action now permits you to request the minimum julia version from your Project.toml instead of having to manually keep them in sink. Announced and future improvements discussed on slack.
  • A new, very opinionated, non-configurable code formatter: Runic.jl
  • Plans to significantly rework the codebase and capabilities of PrettyTables.jl

Soapboxes (blogs/talks):

  • MicroGrad.jl is a wonderful tutorial on automatic differentiation.
  • A new mini-course on Topological Data Analysis with Julia.
  • JuliaHPC is running a helpful reference website for anyone interested in best practices for HPC.
  • Consider subscribing to the French community newsletter (much of the shared materials are in English).
  • Consider subscribing to the community calendar to be informed of upcoming virtual meetings and talks.
  • Consider attending the triage meetings of the julia core contributors (organized on slack) – being a fly on the wall can be a great way to learn the nitty-gritty details of current priorities and development work. These are organized on the triage channel in slack. Minutes are kept.

Please feel free to post below with your own interesting finds, or in-depth explanations, or questions about these developments.

If you would like to help with the draft for next month, please drop your short, well formatted, linked notes in this shared document. Some of it might survive by the time of posting.

72 Likes