A monthly newsletter, mostly on julia internals, digestible for casual observers. A biased, incomplete, editorialized list of what I found interesting this month, with contributions from the community.
JuliaCon2024 happened this month and some of the entries below refer to talks from the conference. Cleaned up recordings will be up on Julia’s Youtube channel in the next months, but for now you can look at the unprocessed live recordings and find the timestamps for a given talk yourself. The schedule of all talks is on pretalx.
“Internals” Fora and Core Repos (Slack/Zulip/Discourse/Github):
- Julia’s capabilities to produce small compiled binaries have progressed significantly. See this PR, and this talk by Jeff and Gabriel at JuliaCon2024
- Julia has a lot of impressive SIMD-focused libraries. At the JuliaCon hackathon work started on support for SIMD intrinsics in base Julia, which can potentially help future work on the aforementioned libraries and more.
- Updating Julia to use LLVM 18 is nearly merged.
- Many conversations on custom optimizations and compiler passes happened at JuliaCon. This work on agressive recursive constant-propagation is one example, mentioned on slack.
- People that compile Julia themselves, now have the option to run pretty aggressive optimizers (profile-guided, link-time, and the new LLVM BOLT optimizer). See the NEWS entries on the topic.
- Exciting work on the
mapandreducefamilies of functions in Julia, addressing a ton of technical debt. - JuliaCon talk: The State of Julia
- JuliaCon talk: Adventures in Julia IR: Plundering Core.Compiler
- JuliaCon talk: Compiler plugins and non-native execution caching
- JuliaCon talk:
Memoryin Julia - JuliaCon talk: App support in Pkg
- JuliaCon talk: ScopedValues – What are they good for
- JuliaCon talk: The
publickeyword and how public APIs work in Julia - JuliaCon talk: BinaryBuilder 2: Return of the Compilers
- JuliaCon talk: Dude, where’s my code? (about code provenance and lowering)
- JuliaCon talk: Julia Compiler Secrets - Static Analysis
- JuliaCon talk: Interfaces.jl: base and package interface tests for your objects
- JuliaCon talk: Writing allocation-free Julia code with AllocCheck.jl
Dustbin of History:
- An interesting discussion on the differences between scalar indexing, non-scalar indexing, and broadcasting, and how there are still a few places where the Julia broadcasting and indexing APIs can evolve.
In search of contributors and new maintainers (specify novice/moderate/expert and internals/domain background necessary):
- The Center for Quantum Networks is running many bounties and minigrants on a variety of support packages for quantum information science.
- SciML is running a large set of funded small projects – a great opportunity to contribute to the community and be paid for it.
- ChainRules, a foundational package to the Julia autodiff effort is looking for a new primary maintainer.
Ecosystem Fora, Maintenance, and Colab Promises (Slack/Zulip/Discourse/Github):
- The journey for Sum Types or ADTs and maybe pattern matching in Julia continues with DynamicSumTypes.jl and Moshi.jl. My read on these development is that DynamicSumTypes provides the simplest way to hook many pre-existing types into a simple sum type in order to avoid runtime dispatch. Such runtime dispatch is frequently the cause of severe slowdown when iterating over arrays containing elements of many diverse types (i.e. a large
Union{...}type) as the compiler and type inference give up. Because of this I think of DynamicSumTypes as “union splitting on steroids”. Moshi is more of a first-principles deep dive into all of the compiler theory and type theory necessary to bring the full richness of a ADTs and pattern matching to julia, with a lot of introspection among the authors about what worked and did not work in previous attempts. - A neat Makie recipe for labeled arrays (arrays in which different dimensions have descriptive metadata) was shared on slack. Implemented here if you would like to avoid the slackhole.
- A slack discussion on how Julia’s BinaryBuilder and the Pkg.jl upcoming app support can be a full-fledged distro package manager, together with a mention of the pre-existing RunBinary.jl and ygg which can be used today for the same purpose.
- Python.jl is an unholy way to run a REPL that is both Python and Julia at the same time. For the naysayers, Python also has a multiple dispatch library
- zotero Slack
printvsshowvsshowwith MIME – how to properly pretty-print in Julia.- JuliaCon talk: Infiltrator.jl - No-Overhead Breakpoints in Julia
- JuliaCon talk: Train a Llama(2) in Julia!
- JuliaCon talk: What’s next for Transducers.jl?
- JuliaCon talk: Updates and Advances in CUDA.jl and oneAPI.jl
- JuliaCon talk: Applications of Distributed Task Parallelism
Notes from other ecosystems:
- A cute little RPG story game / set of tutorials to learn how dangerous multithreading can be.
Soapboxes (blogs/talks):
- Consider subscribing to the French community newsletter (much of the shared materials are in English).
- Consider subscribing to the community calendar to be informed of upcoming virtual meetings and talks.
- Consider attending the triage meetings of the julia core contributors (organized on slack) – being a fly on the wall can be a great way to learn the nitty-gritty details of current priorities and development work. These are organized on the triage channel in slack. Minutes are kept.
Sundry:
Please feel free to post below with your own interesting finds, or in-depth explanations, or questions about these developments.
If you would like to help with the draft for next month, please drop your short, well formatted, linked notes in this shared document. Some of it might survive by the time of posting.