Pardon the delay with this release – end of semester is always hectic.
A monthly newsletter, mostly on julia internals, digestible for casual observers. A biased, incomplete, editorialized list of what I found interesting this month, with contributions from the community.
“Internals” Fora and Core Repos (Slack/Zulip/Discourse/Github):
- With a lot of difficulty to contain my excitement and hype-proneness, it is wonderful to observe the progress Claire is making in her crusade to rewrite the parsing and lowering passes of the Julia compiler in Julia itself, while also producing much tighter, reusable, and introspectable implementation than previously available. Check out the issue tracker of JuliaLowering.jl. In terms of lowering, the discussion on what folks from Racket do is quite informative. Docstrings are also an interesting can of worms. In terms of parsing, there are interesting considerations in making iterations syntax produce better parsed abstract syntax trees (here and here). She also posted a wonderful demo of the new scope-resolution capabilities on slack.
- While functions can be easily redefined “live” in julia (especially with the use of Revise.jl), struct redefinition is less ergonomic and requires restarting the julia process. This is mostly due to functions existing in different “worlds” depending on when they were compiled, while bindings do not have the notion of world and age. This might be changing soon (enabling Revise to redefine structs). See this old proposal and recent (well documented) work on implementation.
- The REPL auto-completion was really finicky for explicitly imported symbols. This is fixed now.
- Handling Ctrl+C interrupts properly is a pain in any language, but Julia is particularly hands-off in terms of how to deal with them (chiefly due to the complication of responding to interrupts from a multi-threaded program). Some interesting approaches to improving the situation were discussed on slack.
- The transition to LLVM 17 is now complete! Thank you Sergio!
- Timothy has created a neat tool to visualize the nanosoldier logs (the tool used to test julia nightlies against the entire ecosystem of packages).
- It is easy to make a struct that does not have a fully specified constructor, i.e. fields of the struct are left undefined. What is the point, being discussed on slack…
- A neat little script that you can run after updating your local version of julia so that all environments you have get precompiled (instead of being surprised when you open an environment you have not touched in a while and having to wait 5 minutes before you can use it).
- A very old well-known performance pitfall in Julia is “variable capture in closures”, due to the fact that the closure needs to check whether the variable was modified outside of the closure. There are standard tricks to get around it most of the time (e.g. with a
letblock to isolate the closure), but there is also recent work on fixing at least some cases without modifying code, here and here. - A very exciting development with a 3rd-party ahead-of-time compiler for julia being made available.
- A generalization of
Fix1andFix2for preparing partially applied functions is being proposed.
In search of contributors and new maintainers (specify novice/moderate/expert and internals/domain background necessary):
- SciML is running a large set of funded small projects – a great opportunity to contribute to the community and be paid for it.
- FixedPointNumbers.jl is looking for volunteer maintainers.
- Northeastern University is looking for research projects to be tackled (with Julia) by teams of their students.
Ecosystem Fora, Maintenance, and Colab Promises (Slack/Zulip/Discourse/Github):
- The Julia User and Developer Survey for 2024 is up.
- A very exciting development in autodiff-world: DifferentiationInterface.jl provides a much cleaner way to interact with the universe of different autodiff tools available in Julia.
- What do various julia autodiff packages promise about second-order derivatives (including nesting of different autodiff packages).
- A very informative discussion on current best practices and capabilities of Julia autodiff packages.
- A new package in the hazy category of “interfaces” and “duck typing” is out, with a really interesting new take on defining (something like) Interfaces in julia. The discourse post also lists the other experimental players in that area of julia language development. However, make sure you check the readme of this new DuckDispatch.jl package, as it provides a lot of useful context as well.
- DispatchDoctor.jl for providing guarantees that your code is typestable. See some of the discussion on runtime dispatch detection that happened during development.
- ArguMend.jl for better, more informative MethodErrors.
- PerfChecker.jl is a new tool for performance tracking of julia packages (mostly to be employed by developers). Similar to AirspeedVelocity.jl and BenchmarkCI.jl.
- A proposal for a new
Structsmodule inBaseto drastically simplify the management of constructors and struct definition. Made available as a registered package for experimentation as StructUtils.jl. - A proposal for a “memory views” API.
- Makie v0.21 is now out.
- MakieTeX.jl is becoming a very capable tool for working not only with TeX, but also with PDF and SVG in Makie.
- A discussion on making test-running more ergonomic and low-latency for package developers.
- UnitfulTensors.jl, a new package for working with numerical data with units, as well as alternatives and competitors, were recently discussed.
- CliffordNumbers.jl, a new Geometric Algebra package.
- The idiosyncrasies of GPU/TPU/CPU-dependent
rand,fill,zero(and the use ofsimilar) discussed on slack. - QuartoDocBuilder.jl is a package inspired by R pkgdown. It provides an easy way to generate a website with documentation and vignettes, by creating a Quarto project.
- CUDA.jl 5.4 is out.
Notes from other ecosystems:
- Alice’s adventures in a differentiable wonderland – a neat new book on a common topic of interest here.
- Preemption and garbage collection in Go and its relationship to Go “threads”.
- A new programming language with focus on GPU: Bend (and their homepage).
Soapboxes (blogs/talks):
- Consider subscribing to the French community newsletter (much of the shared materials are in English).
- Consider subscribing to the community calendar to be informed of upcoming virtual meetings and talks.
- Consider attending the triage meetings of the julia core contributors (organized on slack) – being a fly on the wall can be a great way to learn the nitty-gritty details of current priorities and development work. These are organized on the triage channel in slack. Minutes are kept.
- A fun post on UTF8 string validation (or not) in Rust and Julia, also discussed on hacker news.
- A book on Calculus with Julia.
- Lecture notes for a class on Music Signal Processing with Julia.
Please feel free to post below with your own interesting finds, or in-depth explanations, or questions about these developments.
If you would like to help with the draft for next month, please drop your short, well formatted, linked notes in this shared document. Some of it might survive by the time of posting.
