# Paper on Perceived vs Real Productivity Gains Using LLM

**URL:** https://discourse.julialang.org/t/paper-on-perceived-vs-real-productivity-gains-using-llm/133322
**Category:** Offtopic
**Created:** [October 21, 2025, 7:33am UTC](https://discourse.julialang.org/t/paper-on-perceived-vs-real-productivity-gains-using-llm/133322 "2025-10-21T07:33:25Z")
**Posts on this page:** 1
**Showing post:** 8

<div class="post-metadata">

### Author: ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)
#### Post date: [October 23, 2025, 8:07am UTC](https://discourse.julialang.org/t/paper-on-perceived-vs-real-productivity-gains-using-llm/133322/8 "2025-10-23T08:07:00Z")

</div>

> [@j\_u](#):
>
> P.S. 1: To sum up, I guess the whole situation could be compared to the analogy: is it better to write a paper with a typewriter or a search engine and a computer?

The whole point is that the premise is wrong. It’s like “is it slower to use a internet or a library?” and the methodology section was “search for the thing on Google, flip through and click through all of the links in the Google search results until the answer is found, or try again”. Then the claim is “using a library is faster because we found finding the results could take many pages, or not exist!”.

The reasoning for the issue then ends up being simply due to how mean time is calculated: most of the “internet” results are found in 20 seconds of Googling vs 5 minutes in the library, but in the cases where the result doesn’t exist in the internet, it takes a few hours with this methodology to exhaustively click through all of the results and know it doesn’t exist, while in the library it’s 10 minutes to go 🤷 I guess there isn’t a book on the subject, and the librarian told me so.

Of course, the problem is, that’s not how people search. Or at least, it’s not how you should use Google search and you’re doing it wrong 😅. You google a term, look at the first page, if it’s not there google a different term, otherwise head to a chatroom / Discourse and ask for help (or these days, try an LLM).

You have the exact same issue happening in this methodology. There is a very long tail for how long it can take to finish a PR. I wrote in pretty copious detail, don’t let that tail fool you.

> [@The use of Claude Code in SciML repos](https://discourse.julialang.org/t/the-use-of-claude-code-in-sciml-repos/131009/8):
>
> So it’s pretty public that for about a month now I’ve had 32 processes setup on one of the 64 core 128gb RAM servers to just ssh in, tmux to a window, and tell it to slam on some things non-stop. And it has been really successful!.. with the right definition of success. Let me explain. I think the first will answer the others. Basically, Claude is really not smart at all. There is no extensive algorithm implementation that has come from AI. I know some GSoCers and SciML Small Grants applicants…

If you just accept that some PRs work and some don’t, it takes ~10 seconds per PR. Maybe like 3 minutes if you let it iterate a few times. Any result that says it takes longer than that… what the heck are they doing? For that problem just give up and do it yourself, in which case it just takes a long as doing it without AI.

---

_[View the full topic](https://discourse.julialang.org/t/paper-on-perceived-vs-real-productivity-gains-using-llm/133322)._