That really ruffled your feathers, apparently… Sorry that wasn’t “nice”, but can we please move on? Yes, I am slightly more cautious than you about the (over-) use of LLMs potentially getting away from the human author managing the project.
At this point, it seems to me that you are meeting your responsibilities for the fundamental design, the quality, and reliability of your package. So you can just ignore that comment! I don’t think we’re actually heavily disagreeing on anything! This is just nuances of enthusiasm!
You might be confusing the responses here, I didn’t write the quoted text but I interpret what you wrote here as being directed towards me. Just to clarify.
It’s not banned! For the most part, it’s encouraged!
You should probably also make sure you still have an understanding of the generated code, since otherwise it will be very hard to maintain going forward. But that’s nuances, and not something that can be covered by “policy” to the last detail.
All of these things are about using common sense judgement!
I think I wrote the original quoted text, and it was directed at you in the original thread. But at this point, we’ve established that you’re well within the limits of what I would consider appropriate use of AI, even in that original comment. So I think that point is moot. Besides, the package went through registration and isn’t subject to any further review. Nor am I overly concerned about packages you might be registering in the future.
Anything I and others have mused about in this spun-off-thread really isn’t directed at you or that package specifically (beyond you also being part of this new discussion). In fact, I was more pushing back against the idea in the original post that all LLM usage needs to be declared, like “GM in foods”. Perfectly fine to have a discussion about that. But it’s neither current policy nor in my opinion a reasonable future policy. I think you and I agree on that!
Cool, good to know. I just wanted to clarify that I’ve made my point here, I think that I do have a valid point to make and one to make in opposition to some of the sentiment that has been expressed. We’re good
Of course not! We have plenty of bugs in the ecosystem. Some even argue that Julia has a “correctness problem”. But you can’t expect me or anyone else to find all bugs in all registered packages, or to continue monitoring their quality. It would be good to use AI to find and fix more of those bugs. (Almost) nobody disagrees on that!
But any serious quality issues (LLM slop or others) that I can spot in 30 seconds of reviewing a registration are perfectly good grounds to raise a flag.
I was really only half joking (if that) about labeling or even prohibiting Julia packages that have not gone through an AI review using some state of the art language/coding model. As far as future thinking goes …
You know what, now that you mention it. Copilot has found plenty of typos in my PRs. Not major bugs, just stuff that would be picked up by JETLS, which i’ve tried but nukes my computer’s processor.
It’s also been really good at summarising my changes. It’s like a really thorough, free, somewhat sloppy copy editor. Plenty of false positives but they’re really easy to identify.
I didn’t even think about it because copilot only reviews and summarises my changes, i dont let it implement its suggestions as they’re always either really simple fixes or it generates overly complex solutions.
Once it starts repeating comments i know my PR is in good shape—at least when it comes to code. It still misses documentation editing mistakes but that’s alright for adding new functionality or fixing bugs.
You know, the comparison with GMO food discourse is actually quite apt. There are multiple groups that have very distinct concerns with GMOs. One is concerned about the idea of the technology itself and concept of eating modified proteins, whereas another is concerned about its usage and how it, e.g., enables more use of glyphosates (RoundUp), whereas another is concerned about the control it gives corporations. The first group has made it challenging for the concerns of the latter groups to be taken seriously (or be heard at all).
It’s not about right and wrong, that’s just the thing. None of this is binary or cut-and-dry. Seeking such definitive answers is how this is going in circles.
“Right” – as usual – is probably somewhere in the middle of the more extreme responses. As to whether the general registry will require packages to declare the amount of LLM usage in a package, on a grading scale: very unlikely.
If someone wants to establish such a badge system on a voluntary basis: ok – I’m not sure how much traction it will get, but it’s a fine thing to experiment with.
Should package authors be allowed to ban all LLM usage for contributions for their package, and indicate that with a No-LLM badge? Probably, but that’s an extremely unwise approach, IMO. People should use AI tools to improve the quality of their code. I probably wouldn’t contribute to such a project.
On the other end of the spectrum, if people want to add a MOAARRR-LLM badge: go ahead. Both badges, at their extreme, would decrease my level of trust in that package.
I simply said I would be fine displaying a badge on my project, but I doubt I will get a PR from them. Doesn’t really matter to me, the proof is in the fact that Tachi is undeniably amazing. Sorry to brag, but I think it’s true.
No. See definition of “slop” in Meriam-Webster’s dictionary.
The PR that you’ve submitted is actually a good illustration of the comment I wrote earlier. The “fix” that your coding agent wrote includes a number of hallucinated misleading and incorrect parts.
Just to say you also take a bit something more important, reputational capital.. this is already happened in the past.. when something doesn’t work, often is not just the specific package that is blamed (and the opposite, when something is really cool..). There is some reputation transfer between the individual package and the Julia community as a whole..
It will get no traction at all, unless it is required as a part of package submission to declare the usage of LLM tools. That does not impose on the registry’s users
any particular opinion on the value of those tools, in contrast to a number of
OSS projects that ban any and all LLM-generated submissions, period. That’s
why I think this may be a reasonable compromise.
