Where is Julia heading? Lack of clarity about priorities, long-term direction and governance

Community
,
102

Okay while on the train I had Claude help me finish the list. Let me know if I missed anyone

Julia Lang Repository Maintainers

A comprehensive list of the main current active maintainers of the Julia Language repository (active in the last 2 years), including their GitHub profiles and primary areas of responsibility.

Core Compiler & Runtime

Parser & Frontend

Package Manager & Releases

REPL & User Interface

Arrays & Linear Algebra

Numerical & Mathematical Functions

Memory Management & Garbage Collection

GPU & Parallel Computing

Build System, Deployment & Infrastructure

Multithreading & HPC

Other Core Contributors

Notes:

  • This list includes maintainers who have been actively contributing to JuliaLang organization repositories in the last 2 years
  • This includes the main julia repo as well as other repos in the JuliaLang org (e.g., Pkg.jl)
  • Many maintainers contribute across multiple domains, and the areas listed represent their primary responsibilities
  • The Julia community includes hundreds of active contributors beyond those listed here
  • For the most current information, see the JuliaLang GitHub organization
22 Likes
103

I have an explanation and working example of the use of workspaces (also juliac and the “app” feature) here: Julia 1.12 brings progress on standalone binaries and more [LWN.net]

(That’s a free link. Please consider subscribing to LWN if you’re not already a subscriber.)

5 Likes
104

Let’s get some concrete steps from here:

  1. Let’s try to get a paragraph from everyone for what they plan to do. Standard question to answer: what is in sights for the next year, three years, and 5 years?
  2. Once we have those paragraphs per person, we take that and compile that into a post that summarizes the general trends and add it to the website. This is something Claude is good enough at that it’s no work at all once we have the source material.

@mbauman you think it’s fine if I just do 1 via an issue in the Julia Lang repo that pings people?
@tecosaur maybe update that dev doc page as well?

Once there is a first version people will naturally contribute and help edit, but it’s always about the inertia of getting it off the ground

13 Likes
105

Sounds good to me. I think it’s worth having a page that’s kept (somewhat) up to date. It also helps prospective contributors work out who to approach.

106

As someone who recently drove adoption of Julia within my company I agree that this would be helpful. More or less the way I got a picture of the overall direction of the language was watching the state of Julia talks and then watching talks on those specific subjects. I also monitored the “This Month in Julia” posts for some time which was quite helpful, as well as the podcast. I didn’t mind doing this, but an easier to digest centralized version of these would have certainly been helpful both for my own understanding and for pitching things to my manager. Once a year would have been more than enough for me. No need to make any lofty promises about timelines, just tell us about about the most significant work was done and what the impact is. When we can expect things in the near future is a bonus if there isn’t any uncertainty.

From an industry perspective, it would also be good to know about gaps in the ecosystem being addressed, perhaps in a separate post. A little shameless self promotion, but maybe there are people who would give Julia another shot if they knew there was a much improved gRPC client :sweat_smile:. The core language / standard library is of course important, but sometimes gaps and limitations within the ecosystem are the most significant blocking factor for adoption of Julia in production.

24 Likes
107

I’ve been trying to guess the order by staring at a few people’s organisations, my working hypothesis (totally unverified) is that they’re sorted by date of creation. For example JuliaLang was created at 2011-04-21T06:33:51Z, python at 2012-03-11T15:56:37Z, so if someone was a member of both organisations according to your method they’d belong to JuliaLang, even if, say, they joined the former later. For the record, JuliaComputing was created at 2014-12-21T07:48:54Z, so if my hypothesis is correct (which is a big if) you’ll never see someone belonging to both JuliaLang and JuliaComputing to be listed as belonging only to the latter, unless only the latter is visible (and again, there’s a small technical advantage in making the JuliaLang membership public to be able to run tools which require checking that information).

5 Likes
108

I know, that’s why I was addressing the varying and sometimes conflicting views of the wider thread, not you specifically. I get why it might come across that way because you brought up PEP first and I did go on to specifically respond to points in your post, but other people were also talking about PEP and less accurately.

That is a good thing, and it’s consistent with my earlier specific complaint about what I had to do to learn about JuliaC development. However, this example actually somewhat contradicts the idea that core developer communication precedes industrial investment. PEP 703’s author is a software developer at Meta AI and previously co-authored PyTorch. Meta’s Python projects heavily involve contributions back to CPython; here’s their blogpost touting some including PEP 703. To be clear, I still believe that better communication improves awareness and investment, it just doesn’t only go in that strict direction and specific initiatives can start anywhere.

I disagree with this. An equal or greater amount of hype was raised for the CPython JIT (PEPs 659, 744) at around the same time (I think part of the reason is that “JIT fast” is an easier story than explaining the GIL), and the JIT and no-GIL/free-threaded builds were both introduced as experimental features of v3.13 a year ago. However, these projects did not have the same trajectories. While the no-GIL option was elevated from “experimental” to “officially supported” (PEP 779, also hints at the work needed for a safe default build but leaves it to another PEP), the JIT has so far failed to improve performance (more nuanced developer opinion), and Microsoft troublingly dropped its support for the JIT project and laid off a majority of the team this year, according to a core CPython developer. Promising PEPs and community traction just don’t say anything about the implementation problems or progress, let alone let us infer timelines for practical usage. The PEPs only slightly hinted at the difference between the no-GIL and JIT projects by a deliberate absence (see comment in PEP 774) of a Standards Track PEP for approving or rejecting the JIT outright, but it was otherwise drowned out by the projections and hype in both the informational PEPs and public discourse. Roadmaps are important, but so is tempering expectations for them.

Hard agree, and it’s not just a matter of personal taste. Despite the flexibility of expression, playing audio or video requires more time and data. It’s also far more difficult to rewind and reconsider prior information; even if a podcast goes the extra mile of providing subtitles and an accompanying video with graphics, it’s still much easier to traverse the same information in a text. I’d also heavily advise against automated transcriptions for mitigating these drawbacks because it still won’t have the formatting and brevity of a written text; documentation and release highlights aren’t given in podcasts and slides talks for good reason.

5 Likes
109

Btw typo here, Elliot is not mose’s nickname

2 Likes
110

I think that is magical thinking — new content will not “naturally” emerge, someone has to do the work. Discussions like this always arrive at this point; a long list of things that “should” be happening, and a much shorter list of people willing to do it.

Which is why the people deciding how Julia’s development process works should be the actual developers. If they want more structure, or centralization, or something else, they will figure it out, otherwise, from a practical perspective, there is no way anyone will impose it on them.

The process is already transparent — if you follow Github issues and pull requests, you get an almost complete picture of where things are going, what the remaining roadblocks are, etc. For the rest, if some details is not clear, people usually respond if you ask. What you want is not transparency, but some kind of managerial oversight.

A major complaint seems to be about “planning”. But planning is pretty vacuous without someone who will enforce those plans, and that kind of authority or mechanism is antithetical to Julia’s governance model, and FOSS in general. Companies implement plans by having managers monitor and direct people, and even then, planning software development is inherently difficult.

I actually admire Julia’s development and governance model and think it is perfect for the job. I would describe it as having two related key concepts: consensus and getting it right. Some features mature for years, because instead of a quick fix, Julia’s authors try to come up with a good solution that allows growing the language further, so that they avoid technical debt.

A recent example is redefining structs, which took 8 years to fix. Several workarounds and hacks were proposed and explored in the meantime, and were dismissed as inadequate. The issue was the most grating to my workflow, but in retrospect I am glad that they took their time to arrive at the right solution.

Similarly, while deploying small binaries would be nice in some contexts, I would not want any kind of pressure on the developers, because I am convinced that they are already doing their best, and any intervention would result in inadequate solutions, developer burnout, and smart people leaving the project, which is the last thing we want.

28 Likes
111

So what are you trying to say? Are you suggesting we shouldn’t plan because it is difficult or?

On a personal note, I find the argument that something is difficult to be rather weak. Creating Julia itself was difficult — yet that didn’t stop its founders. Similarly, building more structure and organization may be challenging, but I believe the Julia community is strong enough to make it happen and to learn from other communities in doing so.

It’s not “magical thinking” to say that new content will emerge naturally — that’s already how things work. Creative people constantly make videos and other materials about developments in Python or JavaScript. No one asks them. Some people just enjoy creating material. They might not even be coders themselves. In Julia, for example, Doggo Dot JL created many videos of my packages. I don’t even know who that person is and never talked to them. You may be underestimating how many creative people out there - you only need to give them source of information for their creativity.

No one here has suggested putting more pressure on developers. The idea is simply to improve organization, which would likely reduce pressure, not increase it. Are you suggesting that asking someone making a major change — “what are you planning to do,” “how are you planning to do it,” or “what’s your rationale” — would lead to burnout? I can’t agree with that.

To be honest, I’m having trouble understanding your point. Everyone agrees that planning is difficult, but that doesn’t make it unnecessary. No one is asking developers to take on more work — just to spend a short amount of time outlining their ideas in a centralized way, as many other programming languages already do. We can look at those successful examples, learn from them, and build an even better process for Julia.

Several people in this thread have mentioned that they find it difficult to understand where Julia is heading. If you think that’s not the case, could you please summarize how you see the current direction and what are the priorities at the moment (besides obvious bug fixing, making releases, etc)?

11 Likes
112

Not really. What we want is somebody to collect this into a different format; such that you can get an overview of ongoing major projects without having to spend the considerable effort of following all github PRs, issues and other conversations like slack, zulib, discourse (and having to attend most triage meetings).

I’m not saying that following these primary sources is bad – it’s just a lot of effort. It would be extremely valuable if someone who already follows these primary sources could write that down, including links to primary sources, and crucially keeping the thing somewhat up-to-date.

That will of course be insufficient to get real visibility into the actual projects. But the central overview of what projects exist, plus their high-level descriptions, plus the links to the actual work will allow people who care to delve into the details.

To give a positive example: Java project “valhala”, aka immutable / bitstypes for JVM. Browsing the central list of JEPs (java enhancement projects) allows to see what big things are ongoing. Looking into the specific thing gives you a usually outdated but readable high-level overview. And from that you can start trawling the mailing list archives. (I’m bringing up that project specifically because it’s projected timeline is “when it’s ready, maybe 2050?”, the intermediate write-ups are pretty good, the intermediate write-ups are notoriously outdated, and it is a good example for having to trawl the mailing list archive to get a real feeling for the thing. So in terms of “management / transparency” this is one of the worst examples – and the central structure is still a great boon)

Now, creating that superstructure for the existing disparate efforts takes effort (catalogue efforts, write down high-level explanations, keep tabs to prevent things going completely outdated). I am not volunteering to be that “somebody”.

But the amount of effort is limited: A single hobbyist person could do it (especially by pestering core devs for explanations that then get written down – the person would not need to reverse engineer the high-level ideas from the code of various branches). The monthly julia newsletter is a thing that exists.

Saying that we want that thing to exist is not the same as volunteering to create it. We’re not entitled to get what we want without spending either effort or money.

But expressing that we want that to exist is not nothing, either. Maybe someone will come up to volunteer for this role. If the role gets a lot of cheering / social credit / thanks for the concept in advance, that might make it more probable. Or maybe someone at e.g. ASML decides that spending a 50% FTE position for “julia community organizing” would be a good deal. (I think it would probably be a good deal for ASML – the position of “organizes and makes visible core language efforts” inherently carries a lot of soft power, and this position is vacant ATM)

7 Likes
113

First, it is very frustrating to see people defend the current governance model.

Second, this is not a plan nor a roadmap. You are describing a system that evolves without clear community-driven directions.

3 Likes
114

A realistic roadmap back then would have read, “Re-defining structs is a difficult problem which may require 5 to 10 years to solve, if ever. Don’t count on this being solved when you make long-term plans for your project.”

Maybe that’s the best that can be said in a current road map about compiling small binaries, too. (I haven’t been closely following the developments.)

Such a statement may not satisfy people but is still a kind of transparency, if only to shoot down the constant intense speculations about this feature.

11 Likes
115

I think that most of the time, it is nearly impossible to anticipate when a complex feature will be implemented. The problem is not only complexity, but how volunteers allocate their time, which is impossible to plan or direct.

As many people have said it before in this topic, I think that is implied for all open issues. There are no hard guarantees for anything.

If a company or a project really considers a feature crucial, I think the best option is to dedicate some time to learn about the open issues and PRs and, at some point, start participating in development. That way, they get to gauge the status of the project and will also have someone who is familiar enough with the internals to fix issues as they arise.

It is my honest opinion. I think that the great, durable solutions usually evolve organically.

I am not sure what “community-driven directions” means in this this context, can you please explain? Who is the “community”, and who and how do they “direct”?

5 Likes
116

No one disagrees that this is hard to do. But other communities have learned to make this process more efficient. We can learn from them. Or are you suggesting that Julia is actually doing it right, and that open-source communities like Python, Rust, or JavaScript are doing it wrong — and should be learning from Julia instead?

8 Likes
117

Evidence for this? Every other PL that I am aware of that has lasted has either a central RFC-like process, a BDFL or a company with internal hierarchy calling the shots. In the latter case you know that the best you can do as a community is hope the company is actually listening to devs. In the former case, RFC of some sort (or papers in the C++ world) actually help organize and structure the community discussion, and document a consensus once it emerges.

Some years ago many people agreed that it would be very useful to have something like Rust Traits or C++ Concepts in Julia. Many proposals and implementations sprung up. The topic popped up in State of Julia talks every now and then. There was a now abandoned (?) investigation by keno (GitHub - Keno/InterfaceSpecs.jl: Playground for formal specifications of interfaces in Julia). But ultimately, without any of the core developers taking an active interest and steering things, the lack of process and community organization meant that these activities all fizzled out. My outsider perspective is that other languages are able to profit from the community knowledge and investment much more.

9 Likes
118

But a weaker thing would be already very useful, and is also the only thing we get from python or java or Linux:

People are working on feature “allow redefining of structs”. The feature would look like this, and would be useful for these and that reasons. Difficult parts are this and that. Currently, these parts are out-of-scope. It will probably take 2-\infty years to land (but no promises), from the perspective of 10/2020. Follow one of the following links to see the actual unfiltered discussion / development, and personal blog-posts / hackmd from people working on this. There is no corporate sponsor. This document is scheduled to be updated in 4/2021.

And then, crucially, somebody from either some hypothetical “central steering committee” or from some hypothetical “JWN, Julia Weekly News” follows up with the people who actually do the thing, at some time in april 21 (or there could be earlier updates, if there is awesome progress, or if the people / org working on that abandon the effort).

6 Likes
119

The problem with this scheme is that in most cases, no one is actually “assigned” to open issues. People show up and do the work, in which case progress is actually visible.

“Following up” would mean that once you open a PR fixing an issue, which is stalled for some reason, every now and then someone would ping you on “are you actually making progress on this?” Which is not a major nuisance for a single issue, but a lot of core developers have a many open PRs.

Yes, people do get pinged when it makes sense (their advice is needed etc), but the current community etiquette is that it is used sparingly.

Also, if it comes to aggregating news in the Julia world, Stefan Krastanov’s excellent “This month in Julia world” series is already filling that demand in an excellent way. This is of course news about what already happened (possibly unmerged, but actual code is usually already there), not “plans”, but it is concrete and probably the best way to gauge the pulse of various areas.

1 Like
120

But the This Month in Julia World (October 2025) is not topic oriented. It scatters topics and features in each post. What is asked for by the OP I believe is a central place to discuss main feature development. For example, the juliac + trim, the Memory, the workspace, to name a few I have been aware of.

5 Likes
121

But this is not supposed to be an issue tracker, nor to assign people/resources.

It is supposed to document major efforts that already exist, and possibly document existing currently stalled work on major issues (descriptive, not prescriptive).

Github issues mixe lots of small things with few big things, which makes it hard to keep track. And PRs do not equate to “improvement projects” – typically, such improvement projects span over many PRs where possible.

“Following up”

Following up on a schedule is extremely important. The info needs to clearly state “which point in time does this info refer to”, as well as “when will this info be updated” (and a sensible update is: Nothing happened, not abandoned). Without the “scheduled follow-up” we get a situation where “silently abandoned” and “nobody bothered to update the info” and “not abandoned, but nothing significant happened” becomes indistinguishable. Scheduled follow-up is a keep-alive / heartbeat.

But sure, only do this for major features / efforts, and then follow-up only twice per year to reduce the burden on core people. Or if that is too much, make it once per year.

Something that deserves a big News.md entry / juliacon talk does probably deserve 6 short updates (3 years of development, updated twice a year) that link to existing github / hackmd / whatever discussions. For example, the whole Memory thing.

Yes. It is a huge service to us all.

But it’s not the same as an overview of “what cool things are currently in the works, and where would I get more details (if it were to affect me)”.

10 Likes