hey @JakeGinesin, i saw you cited Reactant.jl so thanks! you guys are the first ones to cite us. we’re talking about doing a first publication on Reactant in a couple of months, but i guess that your publication will go first.
Julia’s lack of software engineering-centric features, lackluster debugging infrastructure, subpar industry adoption, its “rivalry” with Jax, and insufficient interopt all bottleneck further adoption. However, none of these limitations in isolation seem like dealbreakers. On paper, Julia should compete with Python. So again we ask, why doesn’t it?
i think you might want to recheck Reactant.jl for the “‘rivalry’ with Jax” because it literally is like a “Jax for Julia on steroids”. we’re still not there but we’re slowly approaching and allows for interopt with Jax (checkout Calling Python function JIT compiled with JAX from Julia without overhead - #9 by sefffal for calling Jax traced functions with Reactant, and Exporting Lux Models to Jax (via EnzymeJAX & Reactant) | Lux.jl Docs for the other way around).
@avikpal is working hard on making (F)Lux + Reactant a real competitor to Python’s alternatives like Jax and PyTorch
We believe the current state of Julia lacks vision. Julia needs a new constitution: a set of concrete goals for improvement, adoption, and outreach. While Python has a clear list of future goals [2], and individual ecosystems within Julia such as SciML have roadmaps [74], the Julia language itself has nothing but surface-level GitHub issues. It is only once we map out and solve these language-level issues that we can then ask if Julia is capable of succeeding Python as the de-facto language for scientific machine learning.
We ask the Julia and scientific machine learning communities: what is the future for Julia?
this has been kind of discussed other times and the conclusion is roughly that it won’t work. i’m relatively new to the community, but i had a similar impression to you when i started and want to clarify what i’ve learned these last couple of years. IMHO writing a long-term roadmap for the language has several problems:
- it
canwill create false expectations of what’s next to come. just check out what has happened with GATs on rust. some features seem “easy” in the beginning but then need years of work. - we don’t have the answers for many things, so we shouldn’t fix to a “solution”. instead, Julia mindset is more like “let users experiment with different proposals until we arrive to some kind of consensus or we find that it requires language support to work properly”
i feel like point 2 should be better explained. there are topics that are very very complex because they have a huge design space and have seen a lot of heated debate over a lot of years, so we haven’t reach consensus. this kind of topics don’t see much progress because there are a lot of tradeoffs and implementation details to be considered to even take a decision. one of this topics i would say is interface dispatch: should we go on multiple inheritance? should we formalize traits? we don’t know what we need in Julia to make this work properly.
but when after some experimentation some common conclusions start to emerge, then we can start thinking about what new features Julia needs and we can construct some kind of roadmap. for example:
- static dispatch, started from the desire to call Julia generated code outside of Julia without the cost of JIT compilation. a lot of experimentation was performed first in PackageCompiler.jl
- compiler plugins (and custom abstract interpreters), started from the desire to customize Julia’s compiler to write your own compiler transformations, optimizations and codegen backends. it is critical for some communities like GPU and autodiff, but progress has been “steppy”. this one has seen a lot of experimentation (and it’s still considered very experimental) in different packages but check Cassette.jl, GPUCompiler.jl, Mixtape.jl, Enzyme.jl, Zygote.jl…
summarizing, making a long-term roadmap for Julia is hard but short- or even mid-term roadmap is more or less implicitly known by just being around and checking the github issues and prs.
the closest things we have to a roadmap, where you see the progress on the topics i’ve mentioned and more, are the annual “State of Julia” talk at JuliaCon and the monthly “This month in Julia world”