Where is Julia heading? Lack of clarity about priorities, long-term direction and governance

Long interesting discussions. Concerns were raised a few solution were offered and one has ignited into action in Juleps repo. Maybe we set a lock down timer on this thread, it is starting to get repetitive.

Before locking it down if we could find one gracious generous volunteer that shares the same concerns, can help with potential direction of this thread and being named in the Juleps repo, that would be great. We have done enough talking we need action now.

8 Likes

For me the blocker to using Julia, is just-in-time compilation every-time

4 Likes

I agree the discussion becomes repetitive. Certain actions are being take here and here.

2 Likes

In addition to copying Rust’s goal-oriented development model, I hope Julia website can provide greater recognition to those who improve the Julia compiler, as the Rust website does.

It would also be advisable to review Julia Governance page. This sentence appears to be incorrect.

Other members of the Julia community are also actively involved with NumFOCUS in other capacities: Stefan Karpinski is on the advisory council and Logan Kilpatrick is a current board member.

I also hope the blog will give more recognition to the Julia Stewards. Regardless of whether current members renew their terms or are replaced, it would be good to announce it in a blog post.

1 Like

Same, it’s always compiling stuff. Updating packages? Compilation takes forever. Loading packages in a new Pluto notebook or Julia REPL? Bad idea because compilation takes forever. Calling a function for the first time? Better go make a cup of coffee because compilation takes forever. Oops, there’s a typo in a local package I’m using, let me quickly edit it and reload… “Quickly”, you say? Compilation!!

Sometimes I feel like I’m just waiting for stuff to compile most of the time, not writing actual useful code…

Relevant XKCD: xkcd: Compiling, but for me it’s just frustrating

2 Likes

I guess to prove @AMJ right, you know that a complaint thread has run its course when people show up to just start airing random unrelated grievances

35 Likes

Have you used C++? deal.II compiled from scratch took most of the day for me.

1 Like

I have, but Julia feels much more like Python or R (which don’t require compilation, even though they can include precompiled libraries), so I’m subconsciously expecting similar behavior from Julia.

I see C++/C/Rust/Zig code and expect compilation. I see Python and R and don’t expect compilation. I see Julia and don’t expect compilation either, even though I know that Julia is JIT-compiled, I know there’s lowering, there’s LLVM etc. Just a Python-based feeling. But this is indeed becoming off-topic

1 Like

Well said. A spot on analysis of the challenges facing Julia to not on only possibly survive as an obscure niche language but to thrive.
I have little to add except to state my support.

Julia is the first language since mainframe Fortran that I love to use for scientific programming. This having endured C and C++ and gone through Mathematica, Matlab, and Python.

The OP raises very valid points regarding industrial and/or long-term use. The absence of a central Julia Roadmap is a significant problem. The only roadmap I’ve seen is for Symbolics vs Sympy.

On a personal note of self-interest, I also agree with the OP on the pressing need for a static binary compiler.

The rather defensive responses of the “Julia community” here is a bit disappointing.

I wonder if most of the current Julia users are in the academic community wherein many projects are of the research, code, publish, and move on type, typically resulting in abandonware. It would be a shame if the same happened to Julia.

10 Likes

There might be enough to have a (static) compilation split in the thread, but I will try to avoid that side of the thread for now.

In terms of actions, I see JuliaHub mobilizing. That’s great. A roadmap from JuliaHub would be fantastic. JuliaHub has the organizational structure the form cohesive statements composing a roadmap.

For those of us who are not JuliaHub affiliated, I am still trying to figure out what is the structure to pursue. I see that some are trying to layout their individual plans for the next five years. That is also fantastic, but I am not sure if that will actually address the original post in that it is not always clear who those individuals are to people unfamiliar with specific Julia developers.

We do have some organizational structure in that many of us have self-organized into Julia Github Organizations. While these Github organizations mainly are organized around the development of packages, at times certain groups seem to also have needs that influence Julia core development. The Julia Github Organizations do have some degree of visibility and discoverability. While there are not clear regulation of Julia Github Organizations, perhaps there is enough structure there that some of these organizations could produce some kind of development priority statement.

8 Likes

I don’t dispute that there is some value in having a roadmap. If nothing else, it makes people feel better about being in control.

What I do not understand is how knowing what the roadmap is would help me to make a decision about using or not using Julia for a project that starts now. Either Julia has something that I can use right now, or it doesn’t. If it doesn’t, a roadmap is not going to help me, given the uncertainties of whether or not anything is going to materialize at all. If it does, I can use Julia right now. Perhaps I’m not completely happy with what it provides: it doesn’t have all the bells and whistles. But whether those bells and whistles will materialize again depends on that uncertain roadmap.

Things are even less amenable to a rational decision-making if it is a project that will start sometime down the road.

Perhaps I am misunderstanding the motivation?

12 Likes

Any roadmap on improving developer tooling? I could use current julia for many years without upgrading, it’ s just the vscode extension has too many bugs.

2 Likes

I probably don’t need to bump this but … Python started out as a small research language mostly for teaching kids to code. The fact that trillons of dollars and the AI revolution/bubble are riding on it now is a quirk of history, and maybe because they realized they were getting big and needed to professionalize. It didn’t happen overnight though.

I like that Julia is still small, yet powerful. It feels like a lot of important work is happening here and that I can learn from it and ask questions without being washed out in the crowd. It still feels fun. But I also do find myself overwhelmed with choice and a lack of best practices when I’m trying to get to a goal that’s not just learn a cool language. I wonder Makie vs Plots and juliac vs PackageCompiler vs PrecompilationTools vs StaticCompiler and Pluto vs REPL vs Jupyter vs BonitoBook or ProfileView vs ProfileSVG vs ProfileCanvas (the answer is ProfileCanvas). I think this sort of churn is natural and a source of creativity, even if it is frustrating to dive into.

I want to remember some history here, because the grass always seems greener. It’s not like python is a shining beacon of professionalism either. They’re doing they’re best, and at least they try their best to be nice as they build.

I learned python in 2004. I found the classic warts blog post a year later. The year after that, Guido posted PEP 3000 and the python3 roadmap blogpost I assume under community and internal pressure from blog posts like the former. Once they had it ready he posted the very helpful What’s new In python 3.0 summary. It’s concise and good, and I read every changelog for each python version in detail for the next 4 years because I was fascinated and curious. And still even with the Benevolent Dictator doing his best, there was a lot of churn where the core wasn’t listening to the needs of the community. requests came along in 2011 because “frustration” (educated guess here but fixing urllib directly seemed too hard and it was too ossified and there was no vision from the python core to do anything about it, even though it was bad), and requests spawned a series of imitator “X for humans” libraries from the community to address gaps in the python standard lib. tornado and Stackless Python predated those and existed because threading was too slow to run serious businesses on (eventually many of their ideas grew up into asyncio). Nowadays it’s the same story, there’s torch vs tensorflow (torch mostly won, but, you know), the whole mess of trying to navigate statsmodels/scipy/pandas, and pytest vs unittest vs doctest vs nose and build vs setuptools vs conda vs uv vs venv vs pip-tools vs poetry (which all sort of interact and sort of don’t).

I especially wish that python’s core would have declared a roadmap for testing and packaging. There are two competing testing frameworks in core, doctest – elegant but not powerful enough – and unittest – powerful but verbose – and no one I know of uses either of them. Packaging was bolted on late to python (because “batteries included” means only the standard library matters to the core?) with a lot of stumbles, led by the community, through several format that all held onto backwards compatibility for longer than was good for them. It was only in ~2021 that development really got adopted (painfully in some cases) by PyPA, and they didn’t even take over the domain packaging.python.org until, uh, maybe last year, and it contained outdated information that was extremely misleading; and if you went to any python forum and tried to ask about it you would get brushed off with “oh yeah you definitely need to not do it that old way” or there’s this offputting “If you don’t trust pypi don’t use it” comment, or how about this very long but very enlightening thread on the difficulties of evolving a tool everyone uses and examples of pretty bad communication of the core’s vision (which did clearly exist) to the users (who had no idea what was up and were recommending deprecated advice to each other that they had little way of knowing was deprecated). PyPA is better about communicating now but their actual products are still coalescing.

Compared to all that Julia has remarkably consistent vision, and so are several of the subcommunities like ModelingToolkit, JuliaStats, JuliaMath.

But +1 to core dev blog posts to make the vision clearer; the py3k post was a wonderful example that I felt welcomed me in and showed me where the sharp edges were all at once.

39 Likes

This topic was automatically closed after 24 hours. New replies are no longer allowed.