This month in Julia world - 2023-12

Community Newsletter
1

A bit early this month. Enjoy the holidays.

A monthly newsletter, mostly on julia internals, digestible for casual observers. A biased, incomplete, editorialized list of what I found interesting this month, with contributions from the community.

“Internals” Fora and Core Repos (Slack/Zulip/Discourse/Github):

  • Indexing into unicode strings is complicated. Do you want O(1) indexing capabilities or only fast iteration? Do you want to get the byte in the utf8 representation or a unicode codepoint or a self-contained human-readable glyph? Here is a discourse discussion on possible improvements to the indexing API for julia strings.
  • Each function in Julia is the single instance of a singleton type assigned to it. This permits dispatching in higher-order functions and high-performance functional programming (as methods can dispatch and specialize on typeof(func)). The occasional need to construct an instance of these singleton types is discussed on discourse (with links to previous discussions). Relatedly, there is a desire to add an instancemethods function that gives all methods defined for a function when only the function singleton type is available (skipping the need to instantiate the function).
  • A discussion on the #internals channel on slack on customizing the compilation of methods by creating your own AbstractInterpreter. (Julia’s compiler & type-inference infrastructure has been evolving to be much friendlier to external modification, e.g. to support Difractor & Enzyme for autodiff or GPUCompiler & StaticCompiler for generating machine code for “weird” targets).

Core Julia Repos:

  • Compilers frequently need hash tables of simple structs (e.g. containing a couple of integers or pointers). There is a neat super-fast super-simple hashing algorithm used in Firefox that then found its way to the Rust compiler, that is now being considered in Julia (see here and here).
  • A new hasdocs function that checks whether a symbol has a docstring.
  • Potentially a very exciting development in the compiler that might make it possible to optimize away some array allocations (early-stage and incomplete yet, not yet providing speedup). New Julia programmers quickly learn that avoiding heap allocations is one of the best ways to make your code faster. However, arrays in Julia are always heap allocated because the compiler currently does not know how to stack allocate objects whose size is dynamic and not encoded in its type. See this JuliaCon Einhoven talk on custom allocators in Julia for why stack allocations are fast (and usually not even counted in benchmarks) and why heap allocations are slow. This blog post discusses good ways to benchmark these capabilities.
  • Better capabilities to study and profile the performance of the garbage collector in Julia here and here.
  • There has been a longstanding complaint that Julia is using nothing in a way that can be confusing - e.g. a search into a collection might return nothing to represent that the searched-for element does not exist or it might return nothing to represent that the searched-for element is actually nothing itself. This is related to the complaint that Base does not use an Optional type in its API. Anyway, the standard workaround is to use Some and some of the helper methods are getting some improvements to type-stability now.
  • Using ] add … to add a package to a Project.toml now automatically also provides a compat entry as well.
  • Compiling into WebAssembly from Julia is getting better and better. Still experimental though.

Dustbin of History:

Ecosystem Fora, Maintenance, and Colab Promises (Slack/Zulip/Discourse/Github):

Soapboxes (blogs/talks):

Sundry:

  • Start planning about submitting abstracts to JuliaCon 2024 (this time in Europe)

Please feel free to post below with your own interesting finds, or in-depth explanations, or questions about these developments.

If you would like to help with the draft for next month, please drop your short, well formatted, linked notes in this shared document. Some of it might survive by the time of posting.

56 Likes