Thanks for the explanation. In this form, I think it would be beneficial.
However, as it documents existing work (or lack of), it does not address the OP about plans, priorities, making multi-year plans that involve future features, etc.
I think it is already great. I did not say we have to wait, or that we don’t need governance. My point was that the language can grow and function perfectly fine without it, as python has shown.
2 Likes
@bvdmitri – could you chime in whether such a documentation of works-in-progress would address your issue?
I think an important part to get right is the “power dynamics”: One cannot ask core volunteers to “apply for permission” to do cool work, or fill in annoying questionaires / write rigid julia enhancement project reports. If they don’t want to maintain an up-to-date homepage / documentation for their speculative pet project, then they won’t do it.
Instead the structure needs to be: Somebody (I am not volunteering) maintains that page, and asks the actual devs for their input. Some of them might enjoy talking about their work and the recognition of their work-in-progress, and then maintain that page themselves. Others might need to be cajoled for a bi-annual 5 minute update. In some cases the “news in julialang/core” hobby-journalist will need to go through disparate github threads or read commits to half-working branches to summarize the state.
And that’s fine. It “just” needs a capable volunteer with too much free time or an org with a pot of money (ok, this is a big ask, and I would not be terribly surprised if it didn’t happen in the forseeable future).
2 Likes
Yeah, I think that could work. Whatever it is, it needs to be as easy as possible for lots of folks to drop a few sentences in. Other things that folks could easily do include a hackmd document or a www blog pull request (but notifications on the www.julialang.org repo are probably not very strong).
Doing it “outside” of the final document can sometimes feel disempowering — if a curator has to change any language, then it often feels like you need to get a final sign-off again from the devs themselves.
2 Likes
Taking action to Getting a roadmap · Issue #60092 · JuliaLang/julia · GitHub. Here goes nothin’ 
34 Likes
I’m concerned that if each developer writes a short paragraph about their work, the document will become too long and many won’t read it. Therefore, I’m wondering if, to reduce the effort required from developers to create the roadmap, it would be better for each person to list ideas (without descriptions) for improving Julia in the coming years, with a label explaining the stage of development. Some labels could be: “Needs funding,” “Needs volunteers,” “No consensus on design,” “Technically complex,” etc. As these ideas progress, a link with more information could be added.
2 Likes
That’s why the whole point is for that not to be the roadmap and just the information source to build the roadmap from.
That is effectively what is being done in the issue.
9 Likes
I stopped using Julia a while back, and I think this was the crux of it for me:
I was interested in Julia at first because it seemed more promising than Python in terms of room for future growth, but I changed my mind on this after a while as Python kept adding more features like protocols, dataclasses, PathLib, and no-GIL mode while I got the impression that both the language and ecosystem for Julia are “locked in place” at this point. The main thing I’d need to consider it worthwhile to invest in Julia is some kind of explicit promise that new features are coming (mainly traits/multiple inheritance, protocols, and statically-typed AOT compilation), similar roadmaps from libraries (e.g. Distributions.jl and StatsBase.jl on fixing type promotion and out-of-bounds indexing), and generally a much faster pace of development and merging new pull requests (which at the moment tend to stall out or devolve into nitpicking, even for simple bug fixes)—as much as I hate the buzzwords and grift industry that’s sprung up around it, I’d suggest picking up a couple books on agile software development or extreme programming, which dominate the private sector for a reason (they work).
1 Like
JuliaLang/julia averages probably 5-6 merged PRs a day, every single day, for many years. I don’t think this is an accurate characterization unconditionally (maybe it is more true conditional on contributions to certain packages?)
4 Likes
You have to critically look at those ‘features’. Most of those things are just ways to address shortcomings in python itself. They are not features.
2 Likes
We recently had a very detailed one for SciML:
The usual complaint to SciML and package ecosystem is that it’s moving too fast in comparison to some other systems . Reactant didn’t exist more than a year ago so in some sense the whole daily workflow changed. And things like LinearAlgebra just overhauled its API, juliac + trim now exists, etc. So to some everything has change in the last 3 years. But when you say “both the language and ecosystem”, you’re talking about 10,000 different things, it may help to be more specific as to what areas you’re referring to.
10 Likes
Let’s not dismiss new developments like this. Redefining structs, or more accurately reassigning const variables across world ages, is widely considered an important feature, and it also addressed a shortcoming in Julia’s implementation and a deliberate ambiguity in its design. Expanding almost all immutable structs to stack/inline allocation, caching native code in package images, and package extensions are just a few of the other big features that addressed shortcomings.
“Community-driven” isn’t a clear-cut concept to begin with. Some people have gone as far as suggesting a direct democracy of users. Some have a more realistic view of developers volunteering their ideas and time to collaborate with a community of more committed developers, which is the status quo. A community project could’ve originated in and been mostly funded by industry like the Faster CPython team’s JIT but ended up losing that support. Would a more successful industry initiative like the no-GIL CPython build still count as community-driven because it had transparent communications and feedback with the wider core developers?
It’s reasonable to have some sort of publicly visible indicator of a feature being discussed, considered, approved, implemented, or made default, but a straightforward promise that something will happen is basically a lie, not transparency. A feature could be pulled at any status prior to be entrenched in the language specification because of an unforeseen incompatibility or just not living up to our hopes. More transparency about the status of a feature shouldn’t ever be confused with a guarantee.
It’s also entirely reasonable to not use Julia or parts of the wider ecosystem because it lacks the features you need or prefer for writing programs. That doesn’t imply stagnation, in fact 1.12 had a few major changes that I’m still adjusting to even for my relatively casual usage. A language’s trajectory just may not be what you’re looking for. Among the things you named, very experimental AOT compilation for well-inferred programs is on the table now, but strictly speaking, static typing is not being considered.
7 Likes
However, what would be your recommendations regarding medium-term priorities and the long-term strategic approach?
I wasn’t talking about any of that.
My thoughts on this in brief:
I like the idea of the blog post where several julia developers speculate about what they would like to work on or see in the near/far future. Why not.
It would be nice to have some kind of dashboard of what major items are currently being worked on. It’s all technically visible in PRs of course, but there is too much noise there so I know you can’t easily get a high-level picture. I wonder how that could work.
Check out the “julep” tag on github; there are many interesting proposals there. I think one thing we could do to address this is go through that tag and make sure it reflects all the proposals we know about, then make a github project board that tags each with a status (e.g. in progress, incomplete, etc.). This is much more likely to be used and updated if it’s on github where everything is already happening.
Once a year or so, a few of us within JuliaHub discuss our (personal/company) priorities for julia language development into the future. We always manage to do an item or two or three on the list, but by the next year the list is pretty different. So publishing this would not be very helpful, and in fact might be actively misleading. I have seen the word “promise” mentioned in this thread a couple times.That is a very strong word! The community is not in a position to write up a long, say, 5-year roadmap and promise that everything on it will be done.
If you need to peer into the future of some piece of code, a good way is to figure out who maintains that code (e.g. by git blame) and ask them. For example mmtk/relational reached out to the GC maintainers and asked “if we refactored the GC to allowing plugging in mmtk would you accept the PR?”
In open source it is best to take specific, direct actions. For example if you see an issue that you think was closed prematurely, just comment “Why was this closed? Seems to still be a problem. Please reopen”. If you’re wondering e.g. what the status of JuliaC is, feel free to open an issue asking “is this ready to use or should I wait for a 1.0 release?” or “it would be helpful to explain xyz in the readme, please add”.
49 Likes
In the blog post, I would strongly world the 5 year as speculative . Still worth sharing the thoughts but just clear as something that is more of a guide.
8 Likes
Yes, thank you, however, with full respect, this seems to be more about micro structure and a short-term approach (with roughly a one-year horizon) rather than the long-term direction as suggested by the topic, therefore, taking the opportunity, I’d like to ask, are you still maintaining the full frontal, scale focused approach to the “two-language problem,” which now appears to have evolved into a “three-language problem,” or are you instead pursuing a more differentiation oriented strategy?
I know you weren’t, my point is that you’re judging the importance of Python developments in a way you weren’t judging Julia developments, and applying the same standard can easily dismiss some of the biggest improvements Julia has had.
I like a lot of those suggestions and Github is a good place for many things, but I still think something like this ought to be in a separate place from all the other Github issues. I don’t want to sift through a bunch of bug reports, pull requests, or speculative discussions that happen to share the same key words, and I’m pretty sure that’s not what people want to be more visible. I don’t even think it has to be off of Github exactly, but a distinct tag or repo just for communicating the status of active developments would be closer to what people are looking for.
I don’t recognize any developments in Julia today as diverging from the original philosophy of addressing the two-language problem, nor have I perceived a three-language problem. Can you elaborate on what you mean here?
3 Likes
In a nutshell, I mean that, in terms of programming languages, it might be probably wiser to pursue differentiation rather than a full-frontal approach, especially given the limited resources and time constraints.
We can also learn from how VS Code organizes its roadmap. Instead of making definite promises — which, as several people in this thread have pointed out, might not always be possible and could come across as misleading — it clearly states
Our roadmap typically looks out 12-18 months and we establish topics we want to work on. We don’t start with our roadmap on a blank sheet. We develop it based on our last roadmap, the findings we made over the course of the last year, and of course what we heard from you in issues, in face-to-face discussions, stack overflow, and twitter.
You can also see that some topics were never finished. It’s perfectly fine to adjust the initial plan and add or drop some topics. It might be due to a volunteer not available to actually do the work or discovered roadblocks on the way.
When we execute on our roadmap, we keep learning and our assessment of some of the topics listed changes. As a result, we may add or drop topics as we go. After around 12 months we come together to develop the next roadmap.
It also accounts for difficult or uncertain issues that may not be feasible to complete as planned — similar to what @Benny mentioned with the JIT example in Python:
We describe some initiatives as “investigations” or “explorations” which means our goal in the next few months is to better understand the problem and potential solutions before scheduling actual feature work. Once an investigation is done, we will update our plans, either deferring the initiative or committing to it.
VS Code also publishes monthly iteration plans with endgame summaries to track short-term progress.
And before anyone points out that “VS Code has more funding, so we can’t do that,” please note that my suggestion isn’t to copy their approach exactly but to learn from it and bring a similar level of organization and transparency to Julia’s development process. Similar to how we can learn from PEP or Rust RFC.
I also mentioned several times that I believe that Julia core developers actually have rough ideas and plans (thanks @ChrisRackauckas) on what do they want to work on. It would be nice to organize and centralize this ideas in one place. I guess we will do it with ChatGPT after the issue is filled in and make the first blog-post of such form. Hopefully many of us will get somewhat full picture of what is on the table currently.
I do realize that “5-year plan” might be a wishful thinking and is probably not happening, but no need to promise, only outline what is on the table and what are the priorities would be a huge improvement.
15 Likes