My wife is currently in PyCon Greece 2026, and shared with me this interesting piece of work she encountered there:
It is analysis of what (to me at least) seems a rather large dataset showing clear signs of correlation between the increased usage of AI and code quality dropping, at least according to some metrics. To me most of the metrics they use make a lot of sense.
The website is very well put together and the inforgraphics are clear.
I am posting this in the “community” tab because I am interested to hear the experiences of other developers, especially those involved with large-ish projects, whether they have had anecdotally similar feeling, or they feel the opposite of what this study showed.
I am a very small developer with connection to a only a few packages, and I don’t use AI at scale. The only repo I’ve seen substantial AI usage gave me such feelings however. The amount of code contributions made it impossible for me to keep up. Furthermore, I am personally passionate about “good code”, especially relying on existing code, and refactoring code to be reusable, so I am also somewhat troubled by this data.
p.s.: I love GitKraken and I am happy to see this company involved in something like this.
It fits the few small experiences I made until now. My LLM is only allowed to write reports with proposed changes. I am mainly hunting bugs and try to unify a few things. Still the changes proposed often ignore existing code that could be reused and I have to point to that. The same for fixes in general, they are usually more like “lazy fixes” than really fixing the original issue.
Typos and sign errors in formulae are found and fixed nicely though.
Nice to see that my anecdotal evidence is even seen in large code base statistics!
edit: I am a bit unhappy one can only get the white paper by selling ones own soul, though. Research should be openly accessible.
I gave the statistics quoted in that link to Claude and asked it to develop a plan to clean a package up. It came up with several issues to fix - almost all of which were mine. It suggested running Aqua.jl and Jet.jl and it suggested a few other minor changes (eg splitting large files into two or more smaller ones). After a brief discussion between me and Claude, I’ve set it going and I’ll see what I get.
I’m not sure what this means. Doesn’t come as a big surprise, now that writing new code has become easier than ever. I doubt anyone at any of the big corporates will still be writing or even reading code by hand soon, forced by higher powers, management, and economic incentives. A more interesting question is what code quality metrics are still useful if all of the code is handled by robots?
Of course they are still useful, as they largely align with principles of well designed codebase. These principles have been developed over decades, and have been well digested and streamlined over this span. Something not so easy to do within a year or two, especially when things are in so much flux… Perhaps a more contentious question is “should we bother with creating well designed codebases now that only AI reads code”? To which everyone has their own take I am sure.
In my view, you don’t always have to view a codebase as ONLY a means to an end. For some people view this is a collaborative and interactive endeavour, in which they may personally identify and align with. That’s how I view my projects for example. I want them to last, and I want people to believe in their vision, and ultimately share it. People can recreate any functionality of my projects with AI with only a few prompts at the moment, but the functionality was never the only goal of these projects. So I personally care about these metrics.
Aqua.jl and JET.jl are great, but JET may have false positives, so if you instruct an AI to fix all the JET warnings it might end up doing some very strange things. I have found that it can often explain what the JET warnings mean.
I occasionally tell Claude to audit a code base for “maintainability”, or to go to spawn multiple sub agents where one reviews code quality, one the UX design, and one performance, and get the subagents to iterate with each other. These kinds of targeted audits really can do wonders. It will definitely improve the code quality even when the original code was largely “vibe coded” already. I would suspect that Anthropic, etc. would be working on integrating these kinds of targeted internal reviews into their harness. There’s a big incentive for Claude producing high-quality code, so this might very well be an instance of “where things are now”, not where they’ll be in three months. Still a concern, and definitely an interesting analysis
My conspiracy theory is that Claude intentionally adds in a few easy to catch but otherwise mostly harmless code quality issues, so the human has something to comment on and engaged with if they are reading the generated code.