I figured I’d write a post updating people with where we’re at on getting Julia 1.0-alpha ready. We’re several months overdue at this point, so people may be getting impatient. I know that I am and I feel confident that I speak for all Julia core devs – we can’t wait to get this release out the door.
The major work item that’s holding things up at this point is a significant rewrite of the optimizer that @keno is working on. It’s already fairly far along, but still not done. Why is he rewriting the optimizer, you ask? Excellent question. The proximate answer is that we want to change the way the iteration protocol works (motivation,
julep, pull request) and the new iteration protocol is dog slow without a new and much improved optimizer. Changing the iteration protocol to use a nothing | (value, state)
style instead of the current (value, state)
style has revealed that @jameson’s excellent work on supporting these kinds of unions efficiently in the first place got us to this point, but when you push it too hard – which the new iteration protocol does – the optimizer can’t handle it. Hence @keno’s new optimizer, which we needed at some point anyway but were planning on doing in a later 1.x release.
We could have lived without the iteration protocol change in 1.0, however, so the question remains: why are we doing this now, right before 1.0? The real reason that the new optimizer is crucial right now is that we’ve decided on using unions for missing data in 1.0. Now that we’ve started using nothing | value
and missing | value
all over the place in Base
and in the data ecosystem, we’ve found that we’re on the precipice of the old optimizer falling apart anyway. The iteration protocol change reveals this precarious position, for example, when iterating a Union{Missing, Int}
vector: the new iterate
function has a return type something like Union{Nothing, Tuple{Union{Missing, Int}, Int}}
. This just ends up being too much for the poor old optimizer. However the key point is that iteration is just the canary in the union coal mine – now that we’ve embraced small unions, these kinds of nested unions are inevitably going to crop up. Fortunately, the new optimizer handles these kinds of constructs without complaint – and almost incidentally makes the new iteration protocol fast. So the new optimizer is necessary both to change the iteration protocol (nice but not crucial) and to make sure that the new data ecosystem works as well as promised (crucial).
While that’s going on, @mbauman has been making a series of PRs improving generic array code and broadcasting. Every PR that Matt makes is, in my mind, another a significant chunk of array brokenness that we would eventually have regretted that we now won’t have to live with until Julia 2.0. So I’m glad the optimizer work is giving him some time to get it done before 1.0-alpha.
The rest of us core devs are looking around for various things that might be suboptimal, debating if and how to improve them, and fixing them. See the triage label for whatever’s on the table at the moment. Currently there’s only one item in there, which frankly, feels great .
TL;DR: we are very close to being ready for 1.0-alpha and the work that’s delaying it is appropriately important, although as is often the case for programming language development, somewhat obscure.