Keeping Up

obsidianjulua

21m

I cant keep pace with this so almost everything from this point past the dialect is considered vibe coded, my hand edits and gating are actually holding claude back and even a local llm has surpassed what im able to control and edit manually.

Does this mean handing over architecture over to vibe implantation and testing with guidance or how has anyone else dealt with straight being overwhelmed in the field they produce code in, Im asking because my own implementation was beyond llm training since mlir abi marshalling has yet to pick up except with the clangIR and llvmABI but even spending all my time in this I got passed up in a few things and deeply in others, do I just accept those failings and use them on a kind of trust system or spend months to years verifing by hand, or do I just say Im done and let the AI deal with marhalling and any other abi in IR problem by someone else or another vibe coder with more sense… if I had motivation to beat the ai but I just dont see past me paying for what I want to impliment.

How ok is it to know when youve hit a ceiling and allow an AI to implement architecture, is it the same as trusting an undergrad or senior developer with code over time or no?

I was just reading this article; it may be helpful: Rob West’s Website - The Agentic Software Factory is the Wrong Metaphor

Everyone knows that debugging is twice as hard as writing a program in the first place. So you’re as clever as you can be when you write it how will you ever debug it? Brian W. Kernighan and P.J. Pluger, Elements of Programming Style 2d ed. 1978)

That’s the problem with vibe coding getting ahead of native capability. You either have to trust or verify and if the traps you set for edge cases and the like expose a flaw, you have to hunt it down, which can be darned hard.

The Factories We Should Build

“None of this means everything being called a software factory is misguided. There are genuinely production-like parts of software engineering, and standardisation and automation are exactly what we want for them. The build pipeline is one.”

“So a clearer way to frame my argument is this: we should build factories for the parts of software engineering that are actually production, and we should not mistake the factory for software engineering itself. It is also worth noticing that the more thoughtful factory advocates are not far from this position. Warp’s piece says that throughput is now bounded by the rate at which the team can define intent, review traces and approve releases. Defining intent and judging whether the result is right is design work. The bottleneck they describe is the design loop.”

Agents as Design Collaborators

"If agents are designers rather than assembly line workers, the most valuable thing they offer is cheap experimentation. An agent can form a hypothesis about how to solve a problem, implement it, run the tests, inspect the behaviour, revise the design and go round again, at a speed no human team can match. Reeves pointed out in 1992 that software’s economics were unusual because the build was nearly free, which made it rational to design by writing code and testing it. Agents push those economics further: now the cost of producing a candidate design is collapsing too.

That changes where the scarce resource sits. As implementation gets cheaper, the things that remain expensive become proportionally more important: understanding the domain, being clear about intent, choosing the architecture, making trade-offs, verifying behaviour, and deciding whether something actually solves the customer’s problem. Those are exactly the capabilities a factory model tends to push out to the edges, framed as the “input” and the “quality gate”, when they are the core of the work.".

I read this a few times and its very encouraging tbh, What he describes as something that needs a “software factory” is the part Im worried about but the article is also saying the same domain should be a software factory and why.

I did make this hard on myself though because of CI deployment testing and PR merging kinda lets claude identify several standing issues and then deploy cloud and locally to fix things and thats where it gets overwhelming for sure. Its the way though

I rely on it to compensate for my rusty Python and R where my skills have atrophied since I took up Julia, which has a sort of built-in AI because it is so relatively much more ergonomic.

You can see the progression of my AI psychosis/existential crisis, added to the fact that my former job is letting go of hundreds (closer to a thousand than 100) of employees—including myself two weeks ago—in the announcement posts for my package. You can see it starting from this post. I’m writing the more recent one and it’s a full-blown mindmelted treatise.

I’ve not written code in like 2 months. I somehow stumbled onto a ridiculous setup that ensures the clankers do a much better job than me. Now I’m not dumb, i’ve been programming for about 12 years, since my undergrad. Admittedly i’m self taught but i went from a chemistry undergrad in mexico, to software dev in fintech, to a phd at the university of oxford in 3d dislocation dynamics modelling, to implementing gcc’s iso fortran binding standard, to working in scientific computing in a structural biology group for the uk gov. I’m not a super specialist or expert in anything, but i have a very broad knowledge base (thanks 'tism).

My strength was always being knowledgeable enough to be useful but ignorant enough to find interesting, unorthodox approaches to problems. LLMs ate my entire lunch, went home, grew and harvested its own organic ingredients, made an improved version and ate it too in the time it used to take me to slice the veggies, plus it found a better cooking method and improved the spatial layout of the ingredients and kitchen instruments for optimal flow.

In PortfolioOptimisers.jl I somehow created a set of standards, systems, instructions, and skills that have the clankers producing performant, inferrable, correct, general, maintainable, testable, and robust code. I did it because I was scared they would ruin my efforts if i let them loose on something i’d worked on so hard, for so long over 2 full rewrites, thousands of tests and agonising over architecture, API, fieldnames, type and runtime guards, coding and documentation standards, but being very lax with error messages and not being able to keep up with my own docs standards. They’ve improved all aspects of my code, and fixed many composability bugs, increased test coverage, fully documented my APIs, helped me write and keep examples up to date.

Here’s my CLAUDE.md, it points to all the places i mentioned. The systems i mentioned are found in code_health, which i’ll turn into a standalone package at some point. It basically ensures the LLMs write unsloppified, performant and type stable julia code.

In aggregate these things make the clankers write code that is better written and documented than I ever did. I bet i could take these same concepts and apply them to any other project in any language and produce much better code than I ever could. It’s depressing and oppressive, yet encouraging and liberating but i don’t know which side wins out.

Thats what the “Software Factory” paper was about linked early on and its probably what each major developer is facing whether they say it out loud or not, so the only counter is to abuse the tools, make large scale testing and implementation and then the domain expert guides those implantations. You should read that. Sometimes Im glad the code can just be generated but also its sad because that means solving most things in code just require a person and a llm to spend time on it.

Oh i spend time on it, there’s a lot of design choices i make. But the details, implementation, and iteration is up to the clankers. I don’t view it a s a factory, but a workshop. I still babysit the robots and make design decisions, but i bounce them off the clankers. It’s far from “do X, make no mistakes”. But six months ago i was using them to imperfectly implement API docs from existing code. At some point they’ll make better design decisions than me.

There are decisions in software design that are choices that are neither right nor wrong, so there can’t be a “better.” There’s a wonderful book from the mid-1990s – Bringing Design to Software by Terry Winograd. It talks about design being “conscious” and “keeping human concerns at the center,” noting “Real-world problems rarely have a correct solution…” It states that “design is a conversation with materials,” is creative, is communication, has social consequences and, indeed, is a social activity.
All good things to keep in perspective as we transition to a future of working with LLMs to create software.

It’s easy when you are thrown out of a job to take it hard and draw mistaken inferences. It hit me hard in 2008 when an acquisition hit me with the added blow that the market crash had rendered me functionally obsolete, similar to how you may be feeling now. I adapted by applying my sharp legal acumen to different problems. It was long, hard and had a much delayed monetary payoff, actually until this year when I began renting out my lightly used soul in aid of making me obsolete again, this time as a data scientist. Fortunately, it will probably take longer than my actuarial shelf life to play out. For you, however, this may be the first in a rolling series of shocks, so it’s time to readjust. I suggest raising the level of abstraction. Do the meta-programming of QC (which is actually harder than programming in the first place) and conceptualizing new. applications. Your previous experience has given you a hidden trove of explicit knowledge that LLMs can’t capture. So move up the food chain and let the bots handle what they are good at.

I appreciate that but i think in many areas, certain designs are very much better than others. I think that’s the case with software, where small decisions can snowball into large patterns that can change the way you necessarily approach things. LLMs have come to break the old process from both top down and bottom up. Even smaller models are good at changing an established design, provided you have guidelines on invariant preservation via tests. They weren’t great at coming up with good design, but frontier models can now recommend really good ones, especially if you have some guidelines.

Some code they produce is a little weird, like using closures fairly often, but it actually ends up being better than the alternative in all the benchmarks and tests.

The old world is dead and we killed it.

Ill sponser your move to the USA, if you want out just put in and email me the details