JuliaLang/julia is AI slop now and I am heartbroken

Community
1

The commit history of JuliaLang/julia is absolutely heartbreaking to read through. I don’t think it’s worth anyone’s time for me to explain why. If you think that thousands of lines of vibe code per week is sustainable, you do you.

I’ve only contributed tiny bits here and there to the Julia ecosystem, but it’s been my favorite language for many years, and most of my scientific work til now has been done with it. Thus I write this not to change anyone’s mind but because using this language was so dear to me that I can’t bring myself to stop without at least saying goodbye.

Perhaps others will feel the same as I do. Maybe we can get together and work something out.

Until then, farewell.

1 Like
2

I am not yet convinced that Julia is AI slop. Is there some evidence that the language is less performant, more difficult to maintain, or has increased bugs? That being said, I think the general concern is valid. Perhaps it would be useful to have a commit policy (if one does not already exist) that specifies in what ways AI can assist in the commit process and specifies procedures for validation and human oversight.

2 Likes
3

I get the strong reaction to the “vibe coded” commits.
But I do want to give some context from my work with AI.

I see it a bit like a slot machine - you try many times, and 80% is horrible AI slob, but then there’s this really great thing it suddenly produces.
Which explains, why there can be smart people who use AI as a normal part of their tool chain already, and other smart people, who see the slob and think its insane to use it for any serious development work.
This can only lead to a bit of a fight between those two groups, since for one of them it’s a marker of “quality going down” while for the others it’s a normal day of programming with the new flashy AI tool.

That said, I’m pretty critical of AI myself, but I do use it a lot at this point.

I have two modes, one is the slot machine mode, where I put it on hard problems no one wants to work on and which nobody wants to pay for.
If it solves the issue, it’s great! Something important which the whole community had no resources to develop suddenly exists and does real work. If it doesn’t, nobody gets hurt :person_shrugging:
GitHub - SimonDanisch/Lava.jl: THE Julia -> SPIRV compiler, which allows to replace all Vulkan shaders with julia functions · GitHub is such an example. It’s 90% vibe coded, and likely a real mess, but it solves a very concrete gap in the current ecosystem, and I’ve been trying to work on this for ages, but it has been just too big and “expensive”. Now it’s there and I can prototype with it and try things out, which is infinity times better than not having it at all. So in other words, horrible vibe coded AI slob doing really cool things!

The other mode I use AI for is debugging and writing small fixes or refactors, where I look and check every line intensely and also make sure there are tests for it etc.
The chance that slob comes out of that mode is pretty close to 0.
My trust in the Julia developers and quickly looking at the claude code commits does make me think, that those commits fall into the latter category and are totally fine!

Even more so, I do hope this gets more!
Julia is seriously underfunded with lots of concrete problems that need a ton of time and polish which literally nobody pays for or does in their free time.

Using AI to fill those gaps whenever possible, will likely shift that quite a bit, and I have big hopes that some of the worst areas in the Julia ecosystem will have a chance to finally get the polish they deserve (developer tools, debugger, performance issues, TTFX, deployment etc).
So, sane use of AI in my mind will be the only way, to finally get the polish and improvements, that have been long wished for by the community - unless of course some huge company suddenly decides to put many millions into those large funding gaps. I don’t see that in the near future, so I’m happy that we get fixes and improvements from smart use of AI :slight_smile:

37 Likes
4

I am strongly against this kind of police work. Who’s going to police that? How can you prove you adhere to the policy? What if someone starts accusing you? Note that just having claude being a co-committer is not enough, in fact it can be wrong both ways. I can write code myself, and tell claude to produce a set of commits. But also the other way around, I can let claude write code and commit the changes myself.

Totally agree. There are gaps and low hanging fruit everywhere you look. Even many of the commits that OP accuses of slop (without evidence) are done by volunteers who spend their time on the project.

7 Likes
5

And that company would still use AI. Unless we migrated to another planet.

One thing to note is that Claude appearing or not as a contributor depends exclusively on how the commit was uploaded. I’m pretty sure it os being used much more than what commit histories indicate, here and everywhere else.

8 Likes
6

Reviewing code (or I guess policing it) is common for large projects. GitHub has many tools to facilitate and document the review process. The purpose of my suggestion was to have a policy for the sake of transparency, even if the policy is that there are no restrictions on the use of AI and no one is required to review the code before it is committed.

7

I think a maintainer needs to approve any merge request to the julia registry. You should apply for a maintainer spot and handle the PR requests.

8

Just to put things into perspective, a similar topic arose 3 days ago on the Python discourse.

To me, if the PRs are properly reviewed by humans, I don’t see any reason to not use AI to improve the language and the tooling. It can cover up the lack of funding.

11 Likes
9

I don’t think you intended to be insulting, but I think this post nonetheless comes off as a bit rude towards JuliaLang/julia contributors who use AI responsibly as a tool by characterizing their contributions as “slop”

12 Likes
10

I agree with Maucejo that the crux of the issue comes down to quality control. The code will either work or not work, regardless of who or what wrote it. However, I think having a policy is good practice because it increases transparency about the review process and level of rigor one might expect.

I don’t think I am qualified to review commits to Julia, but would the current policy (written or not written) allow me to review PRs? I could simply accept a PR without looking, or without any expertise to guide my review. Ultimately, I would like to know what the policy is so I can have a basic understanding of the quality control process.

2 Likes
11

yes, PR review is accepted and encouraged! although please don’t do it “without looking” :slight_smile: it is only a useful signal if some actual effort has gone into the review.

1 Like
12

There’s a policy already for packages in general: GitHub - JuliaRegistries/General: The official registry of general Julia packages · GitHub

2 Likes
13

Julia itself does have such a policy for contributions

  • If your pull request contains substantial contributions from a generative AI tool, please disclose so with details, and review all changes before opening.
12 Likes
14

The interesting aspect of such a policy is that it requires to disclose with details the use of AI tools, and that can make the Julia language appear to be more developed in this way than other languages, which could lead to the reaction of this post for critics. So the difference between Julia and other languages could be a matter of disclosure policies more than anything else. Transparency is bad :smiley:

3 Likes
15

On the flip side, the transparency here also allows you to see all the review that’s gone into these PRs. Folks may get a different impression if you look more substantively at what’s happening.

5 Likes
16

The title of this thread is needlessly divisive.

It is clearly not, or at least not anywhere near what we define as “slop” in the context of the LLM guidelines for packages registered in General. Let’s try to have a more nuanced conversation and not go down the same road of extreme AI-denialism vs AI-is-the-100x-engineer as in the recent Policies around registering AI-created packages in General? thread.

I would trust the core developers of Julia, who are all extremely accomplished and experienced software developers to be able to avoid many of the pitfalls of unsupervised AI overuse. It also seems clear to me that Julia has many rough edges that are due to limited developer resources. Agentic coding can be an enormous help in helping with at least the low-hanging fruit.

I certainly understand having reservations about LLM overuse, but denouncing entire projects just because they make use of agentic coding is throwing out the baby with the bathwater. How could they not? These reflexive responses to LLMs usage, “saying goodbye”, or forking a no-AI version of vim seem utterly misguided to me.

As for “Claude” co-authoship: that, at best, is a way of managing sandboxed access for the agent, or at worst, a fad for deflecting responsibility.

All that being said, it is of course useful for the Julia project to document development practices, including the way that agents are used by core developers. And, of course, placing scrutiny on submitted PRs that there are human authors that can stand behind the code. julia/CONTRIBUTING.md at master · JuliaLang/julia · GitHub seems to go a long way towards that, and doesn’t seem to indicate me a need for panic.

16 Likes
17

Thank you for the good faith. I use language like this not with intent to insult but rather to express degree of emotion. I want to clarify that my concerns are not about the quality of generated code. I do not think it is possible to use this technology responsibly. I have tried.

As a programmer, I find the practice of offloading critical thinking to a neural network rented from a datacenter erosive to the collaborative and educational spirit of free software. As a scientist, I find it derogatory to the scientific method itself. As a citizen, I find it politically immature.

Perhaps the community does not generally share this perspective. Carry on, then. Be well.

14 Likes
18

I share some of your concerns, but, it is not impossible to use these tools well in many cases. I think that generally speaking it is true that people necessarily understand less generated code, but it is not 100% to 0% understanding. And more you review, more the percentage of understanding increases. Just as when you read a complex theorem, and re-read again to improve understanding. Writing it in the first place could be in some cases undeniably harder and require more understanding though.

The enforced policy in Julia tries to strike a reasonable balance between total ban and total unresponsible use in my opinion. Maybe it could be expanded a little bit though.

19

If you had written something actionable or concrete, we could have had a constructive discussion. Except your first post is to call the julia language itself slop? I am confused about what the purpose of your post is.

20

I think people need to embrace the future and stop being so extreme. AI is a tool! A tool that can help us be extremely more productive. However, as any other tool, you can also make terrible things faster too.

To sustain your point, please point out any merged commit to Julia code itself or any important package in the ecosystem that you can classify as generated by AI and garbage.

On the other hand, I can point out many, many merged commits with very good quality that solved useful problems.

I am a developer who has been trying to contribute to this community for more than 13 years on my free time. If I can use a tool to do the heavy lifting for me so that I can be more productive and help the community, I will use this without blinking.

8 Likes