I think that Julia discourse should host some discussion about the use of LLM
and coding agents in the registered packages.
Sure @Mikhail_Kagalenko, what sorts of topics are you interested in covering?
I think that all packages that include neural-net output should be clearly marked
in the Genral registry, because some are opposed to them as a matter of principle.
2 Likes
@Mikhail_Kagalenko Ok, how would you like them to be marked?
I would like a banner similar to the CI testing status, so
I can tell those packages at a glance. A bit like
the requirement to warn a consumer about GM-modified
foods.
2 Likes
@Mikhail_Kagalenko Would you like to submit a PR to the README.md file with a proposed badge?
1 Like
That wouldn’t make sense if implemented just for a single package.
I think that there should be a requirement in General registry
to clearly mark all “AI” coded packages as such.
I don’t think that’s going to happen. We have a policy against vibe-coded packages, but it’s neither realistic, useful, or enforceable to ask that “no LLM has ever touched any part of this code”. I would expect there not to be any actively developed packages in the near future that would qualify for this. People are of course encouraged to disclose in some form of their choosing when they use LLMs as a very significant part of their workflow. But at the end of the day, our expectation is that humans have ownership their code. You can use an LLM, or any other automated tools such as linters and formatters, to write code or tests or documentation on your behalf, but you’re responsible for whatever you commit.
9 Likes
I think at this point you will struggle to find packages that have been created without any AI input. So you will have to mark all of them, which is not very useful.
2 Likes
I offer the code in Tachikoma.jl free of charge, under the MIT license which states:
THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
If anyone thinks Tachikoma is too “vibey” for them, I’m ok with that. But I am glad to know that we can use tools, like linters, formatters, IDEs, compilers, etc. etc., and not still be using punch cards. 
I think it’s pretty cool, personally, I’m building a lot of other projects on top of it, many of them targeted at making the “vibes” even better!
I also hope that there’s at least some appreciation that the author of the code, me, human, spends time responding to questions, bug reports, pull requests, feature enhancements, etc.
Have fun with it!
(Oh yeah I pushed a new version which fixes some things, so go try that out. I’ve been using Tachi to build other projects and wanted to provide hooks so that one could have essentially hot-code reload through Revise.jl. My vibe coding assistant can make changes and they just pop up instantly without need to restart!)
7 Likes
I think that most people are well-meaning and will respect the disclosure rule.
Those few that disregard it are likely to break the already existing policy
against the “vibe coding,” and are going to be sloppy and obvious about it.
Of course, it’s not realistic to fully enforce it, but that goes for pretty
much any law or rule.
Who can predict the future with any certainty? It’s rather common
observation that the “AI” industry is an unsustainable speculative
bubble that is going to burst soon.
1 Like
If true then I’m glad I’m getting some utility out of it while it lasts! 
1 Like
Absolutely agree. A lot of people are going to lose a tonne of money on this. However, the coding assistants we use today are not going away and will continue to be more and more common. I personally don’t view coding assistants as AI at all. To me it’s a clever development tool that can take care of tedious stuff like refactoring etc.
Thus, I also don’t really see the utility of the badge. Instead I think it’s better to create a GitHub/GitLab account for your assistant so that it gets the credit for the precise code it touches. Maybe Claude, Vibe, and Codex will all be there 
4 Likes
Even that seems like a fad to me of people anthropomorphizing their LLM agents. After all, I don’t give credit to JuliaFormatter or JETLS, either. As you say, it’s just a code editing tool, albeit one that’s a bit more powerful than what we’ve had in the past. I understand the amusing novelty of having a “Claude” committer. I wouldn’t, personally, and avoid letting agents create commits. That commit step seems like a good point where I should be reviewing the work as a human, and that also naturally avoid the “attribution” concern. The agent (despite what some people have deluded themselves into) isn’t an independent intelligent being that warrants any such thing as “attribution”. I see the attributed commits as more of a cop-out, akin to “Sent from my iPhone” (Don’t yell at me for typos). In this case “Don’t yell at me for slop”. When, of course, that is no excuse: people should be yelled at for slop (or typos).
P.S.: I don’t think my opinions actually diverge that far from @kahliburke: Use LLMs as a tool, but own the result. And yes,
That’s more than appreciated!
I do think this package is on a fundamentally good path and seems to use LLMs judiciously (just like the use of Claude in SciML projects)
8 Likes
The beautiful thing about a discourse board, @Mikhail_Kagalenko, is that you needn’t ask permission! I’ve split this more general discussion around General’s policies to its own thread, leaving some of Tachikoma’s specifics in its announcement thread, but in the future you can also just create new topics yourself 
My apologies if I disrupted any of the discussions, but I think I found a sensible split between the general on General (here) and the specific on Tachikoma (there).
7 Likes
For the people who find this important, it would be better to voluntarily make a badge saying your project is AI generated or not. As for me, I would just ignore the badge and review the package the same way as I review any other project: looking at how the project is run, the quality of the code and it’s organization, documentation, tests, CI/CD infrastructure, etc.
2 Likes
On reason I did not yet try the package this discussion spun of (Tachikoma) is that is reads quite vibe coded. Nothing against you, khaliburke, you seem to answer a lot of questions and care about fixing stuff; still this is a bit too far out in “AI is awesome, I use it to make my coffee”-land for me.
As a side remark to this, I recently added a paragraph to my CONTRIBUTING.md stating
and for a badge, one could maybe introduce some kind of levels (and leave it to the author to decide where they see themselves
- A no LLM contribution whatsoever
- B some help allowed, when it is used to rephrase docs and such easy tasks, but nowhere critical
- C LLMs allowed but a user has to state where and when
- D LLMs allowed
- E vibe coding allowed
I would be in B (yes these 5 are just what came to my mind right now)
1 Like
You mean you still make your own coffee? 
So if an experienced, knowledgeable user makes a contribution in a critical part of the code but used AI, you would deny it purely because of how it was created? I find this quite bizarre to be honest. It seems that in this whole discussion it is never about the code, but only about how it was written.
There’s a recent blog post by the author of tldraw that I think is important reading material: Stay away from my trash! - tldraw: Infinite Canvas SDK for React I’ll just cite the last bit here:
Where this leaves us: This leaves us at a place where the best thing to do for our codebase is to shut down external contributions, at least until GitHub offers better support for controlling who can contribute, when, and where. Assuming the social contract of code contribution remains unchanged, then we need better tools to maintain the peace.
But if you ask me, the bigger threat to GitHub’s model comes from the rapid devaluation of someone else’s code. When code was hard to write and low-effort work was easy to identify, it was worth the cost to review the good stuff. If code is easy to write and bad work is virtually indistinguishable from good, then the value of external contribution is probably less than zero.
If that’s the case, which I’m starting to think it is, then it’s better to limit community contribution to the places it still matters: reporting, discussion, perspective, and care. Don’t worry about the code, I can push the button myself.
I like your new contribution section since it doesn’t disallow AI, but at the same time, asks for thinking about the result before asking for review. In general, though, I don’t think that the levels badge make sense. It is clearly possible that an “A” contribution is worse than an “E” contribution. In general, the main thing I would ask to other people is not to suggest to merge code which they didn’t care to try to understand or check fully, responsibility is the main requirement in my opinion, not that the code is generated or not. This also means that a contributor needs to spend more time reviewing than writing, more the code moves across the levels you mentioned…I would say some sort of effort one way or the other is what is needed.
1 Like