Reliability of AI coding tools

Before moving on to package-related updates, and since the parallel thread had a timing that didn’t allow me to respond earlier, I wanted to address one constructive piece of feedback from @BeastyBlacksmith here it is:

"This is exactly the kind of constructive feedback I was hoping for, so thank you very much for taking the time to write it.

Let me describe my own profile, which I think overlaps quite a bit with that of many people who use code primarily for research or day-to-day tasks. I’ve been coding for many years, always in the context of data analysis and ML model development, all within computational linguistics. My time is very limited and code is a valuable tool rather than an end in itself. I used to work mostly in Python and R (and still do for certain tasks). I like Julia because of the freedom and expressiveness it gives me. I learned it in small pockets of time a few years ago. But the relatively limited ecosystem, compared to other languages, meant I couldn’t afford to invest the extra time to “bridge the gaps” myself by writing the missing tools and packages I needed.

Then coding agents arrived and last year I started developing my first Julia package, TextAssociations.jl, using Claude Sonnet 3.5. For that package, I used very little agent-generated code. People who browse that codebase will see that the traits you mention—hardcoded repetition, lack of abstractions, etc.—are not really there; there is little hardcoded code and quite a few layers of abstraction. This was my first Julia package and I already see it as something I can introduce to my students so we can build a small team who learn the language and start writing Julia scripts for text analysis.

With Swirl.jl, things had to move much faster. My main motivation was for students to be able to use the REPL to learn Julia at their own pace, so that we could save time in class. I had the idea, I knew roughly the structure I wanted for the package, but I had no time to scaffold a new project from scratch. So you can imagine my surprise when, with a few prompts, the system produced something like 70–80% of what I needed. Even then, I had to intervene many times and spend quite a few hours to make it actually work. I deliberately ignored “ugly” code that worked and focused on what didn’t work, or what needed to change to be practical for my use case. After the announcement, I received many encouraging comments that gave me the motivation to invest more time in making the code a bit more functional (thanks for the issues you opened in any case..)

The rhetoric you describe around the current AI hype was quite new to me in this context. I never meant to “steal” anyone’s code (whose code would I even be stealing, in this case?) or to show disrespect for developers’ work. On the contrary, I was happy to share Swirl.jl with people whose coding creativity I admire and for whom I have a lot of respect.

That’s my story, and I’m fairly confident it overlaps with the stories of many other people. On the positive side, there has only been one genuinely aggressive reaction so far, which is not too bad given the kind of debates that you say that currently surround AI and coding tools.

To sum up, I believe developers can only benefit from seeing domain experts from other fields engage more deeply with their tools and ecosystems—rather than feeling intimidated or ‘robbed’ by that participation. Discouraging people from building their own Julia packages is counterproductive; it risks pushing them away from adopting the X language (julia in this case) altogether, or using it silently without ever feeling welcome enough to return to the X Discourse community."

12 Likes