Synopsis: An MIT researcher set up an experiment that used an electroencephalogram to monitor people’s brain activity while they wrote essays, either with no digital assistance, or with the help of an internet search engine, or ChatGPT. Result: the more external help participants had, the lower their level of brain connectivity, so those who used ChatGPT to write showed significantly less activity in the brain networks associated with cognitive processing, attention and creativity.
Take home: Using LLMs will probably have a negative effect on our cognitive function in the long run.
I don’t get it. If the response is correct, why you care about where it comes from? Sometimes, we use LLM to generate texts simply because we are not native English speakers.
in this case, the response was not correct, nor would I even really call it a “response” to the questions raised. it was just several hundred words of AI slop from pasting every issue comment into a chatbot.
So that is a single case and very specific one. But it seems to me it is quite harsh to draw general guidelines from such a special case. The AI generated code/response is going to be more and more popular. Even SciML adopts it to fix issues as described by Kris.
Sometimes, we use LLM to generate texts simply because we are not native English speakers.
but you would be using the LLM to translate from your native language. you are really generating the content of the response yourself. I think the original comment was meant to dissuade just copying review comments to an LLM and pasting the response back (without revision, human input, scrutiny, etc.)
The suggestion to explicitly ban LLM comments in threads related to a package registration is in reaction to a specific past incident where a question was raised along the lines of “how does this new package relate to specific existing packages”. The package author took that question, pasted it into Claude and replied with several thousand words of Claude’s response. That response was mostly AI slop, and lacked any understanding of the subject matter.
Obviously, that was extremely unhelpful. This addition to the guidelines would just make it explicit that such a response is not acceptable and could be taken as grounds to not further interact with that author or that registration.
(If I can’t tell your comment is an LLM response, you’re fine. By all means, use LLMs to get around language barriers or to help yourself think through an issue)
There’s a huge difference between making an LLM generate a response to another human and making it automate simple code fixes that get thoroughly tested. There’s also a huge difference between making an LLM generate a response to another human and having any AI or other software translate human responses.
You might argue that translated texts count as being generated in a broader sense, but that obviously is irrelevant to the guidelines trying to curb people essentially running bots under their name instead of paying attention. That doesn’t even require AI; if I were pasting other people’s opinions under my name and I don’t care to vet it for the conversation, then I’d deserve to get removed for undisclosed sources and spam. LLMs not only made this easier, its marketing actively encourages this for business communications. That may fly for checking boxes in some dysfunctional corporate environment, but not for overworked developers offering their time to listen.
Since you’re talking about machine translation, it’s worth informing people how you are translating and what languages are involved, and everyone involved should keep in mind that automated translation is not a solved problem. It’s also worth looking into other options. LLMs do great at sounding fluent in casual conversations, but the hallucinations are often terrible for technical conversations, where an otherwise correct response with wrong word choices and grammar would be preferable to fluent falsehoods.
Thanks for the useful discussion everyone! It seems like @goerz’s PR Add LLM guidelines by goerz · Pull Request #140718 · JuliaRegistries/General · GitHub is a good next step to try here, but we’ll probably have to iterate as things evolve and we see how well it works in practice. If anyone has specific feedback/refinement for that PR, please make it over on GitHub so the discussion can be consolidated there.
If anyone wants to get involved in building a tags system for General, that also sounds like a useful follow up that might be able to address curation concerns and licensing concerns all in one. I think probably the next step there is to do some research on what other package registries do, and the pros/cons for their approaches. If that topic interests you I’d encourage you to start a new thread with some thoughts or targeted review on the topic.
I had a student that didn’t speak English well. Once he sent me a large email written in flawless English asking some physics question. But the email didn’t make sense, I couldn’t understand what he wanted to know. I told him that this email looked like LLM text, and asked him to write his question in his own words. He admitted it was, and explained his question with a few sentences of broken English that were easy to understand.
This phenomenon is well-known, it’s called “lossy expansion”. So please don’t use LLMs to generate your replies even if your English is bad.
I am not sure which model he was using. But, with Gemini Pro 2.5 and proper prompts, I see no serious the so called “lossy expansion” occurs. It works pretty good to translate my Chinese into English with exact the same meaning.
That’s great to hear! It’s definitely true that the capabilities of these models have improved dramatically.
You’re highlighting two really important factors:
Model Quality: Newer, more advanced models (like the latest versions of Gemini) have a much deeper understanding of language, context, and nuance. This makes them far better at preserving the precise meaning, tone, and even cultural subtleties that are often the first things lost in translation.
Prompt Engineering: This is key. How you ask the model to perform the translation makes a huge difference. Providing context, specifying the desired tone (e.g., “formal,” “casual,” “technical”), or asking it to “ensure the exact meaning is preserved” can guide it away from a simple, literal translation and toward one that is truly equivalent in meaning.
The old “lossy expansion” idea probably comes from earlier machine translation days, where systems would just swap words or simple phrases, losing all the underlying intent. It’s encouraging to see that with the right tools and techniques, we’re moving well beyond that.
Thanks for sharing your experience! It’s a good data point for everyone.
Prime example of an LLM making something up that obviously contradicts the actual discussion.
I think this is just basic social skills, not just in our community. 30-minute monologues and 1000 word walls might be fine for one-off responses or reports, but it obviously doesn’t help live conversations over specific points. There’s no possibility of such long generated responses even accidentally being correct for forums like this or Github.
No offense intended; if someone is always capable of reading and writing English with the exact meaning they can in fluent Chinese, then there would be no strict need for machine translation because they are fluent in English by definition. Realistically, we must consider people who are sometimes less certain of the meaning of a text, perhaps illiterate, in one language than another. I think these are reasonable expectations:
All parties must divulge any use of machine translation or text generation software in a visible place in the discussion. I don’t think it’s always necessary, but a conservative approach would be to preface each message.
Never post responses simply generated from other people’s words as prompts; this is tantamount to running a chatbot in your name, in which case you’d better direct-message them a suggestion to use the chatbot themselves instead of taking up space in the public discussion. Translate their words to your favored language, write a response you fully understand and agree with, translate your response back to their language. Factcheck and edit each version of your response for accuracy to the best of your ability before posting, though that’s reasonably limited for the foreign language version.
If openly specified machine translation is being used in a discussion, participation is an implicit acknowledgement that the language barrier may be insurmountable and an agreement not to blame anybody for being unable to speak any language. Just give up politely in the case where the discussion can’t progress.
These principles don’t solely apply to translation. I’ve seen experienced Julia users post wrong comments about Julia fundamentals a few times, and the chatbot wasn’t always divulged before getting factchecked. They were proven fluent enough in English to factcheck or edit the response themselves, so there wasn’t a good excuse to neglect doing so in the first place.
I would go as far as arguing the improved tooling ([sources] etc.) means we could reasonably apply that bar within General, but I think by this point that horse has bolted.
I’m not sold on this idea. To me tags seem like an orthogonal feature (but helpful generally!).
Just speaking for myself, what I yearn for is a curated subset of packages that only depend on other curated packages, and are helpful for discovery. I don’t imagine tags would allow for this style of layering.
Figure 1: The ‘registry’ structure I currently find most appealing, with arrows representing dependency.
I know that Pkg.generate is intended to be quite minimal, but I have been thinking it could benefit from a little spruce-up, without going the full way of BestieTemplates.jl etc.
This is an issue related on how much people rely on something, not only for LLMs. If someone tells you that you can trust about something and the thing simplify and fasten your works anyone take that shortcut, otherwise if you instill doubts they tend to asking yourself if that thing is cheating on you or is reliable.
Just speaking for myself, what I yearn for is a curated subset of packages that only depend on other curated packages, and are helpful for discovery. I don’t imagine tags would allow for this style of layering.
Sure, they would . They would need to be registry-managed rather than user-managed. E.g. a package-version could qualify for a tag if it meets a certain bar and all its deps do. And we could do things like pause automerge if a new package-version was going to lose its tag for some reason (new dep doesn’t qualify). And if tags were queryable from Pkg then we would have discovery.
This is a valid concern, so let me revise my proposal to address this: all packages remain in the registry, and manifests (which use UUIDs) remain installable.
When a package is removed, all that changes is the mapping from name to UUID: either the mapping becomes unresolvable, or it will point to a different UUID. This, of course, requires changes in how Julia handles registries, and new features for registry formats etc. This has been done before, eg the introduction of new registry flavors in Julia 1.8.
So, as a thought experiment, suppose that this feature is introduced in Julia 1.14, and consider BadPackage.jl which was removed from the registry.
In versions up to 1.13,
add> BadPackage
would work just fine. From 1.14, the user would be told that
add> BadPackage
ERROR: the name BadPackage was deprecated in this registry because of [reason].
Use explicit UUIDs to install.
Pkg.instantiate (from an explicit Manifest.toml) will continue to work fine, without any warnings.
