That said, the limitations are kind of obvious and are starting to show in some of my projects, and this article seems to confirm my suspicions. If it's just confirmation bias or not, I can't say yet.
In my experience, for anything complex enough, I have to start adding more and more constraints, style guides, corner cases, error handling, optimization guidelines and all this good stuff to my Markdown specifications, rules and skills. At some point this starts to look like we're all just moving complexity from the more formal and deterministic world of programming languages to the informal and non-deterministic world of natural language. The writing speed gains are enormous, yeah, and business sees this as productivity gains, of course - and we do it because the pressure for increased productivity is there, as it's always been; yet the trade off seems to be clear and a lot of people are just ignoring it.
This is the problem nobody is talking about. I see codebases growing in MD files with instructions and guidelines and requests that are also LLM generated… and it’s all piling up. No one is reviewing it 100% , and even when we do, it’s all very subjective. What’s the difference between “Follow a RESTful approach”, “We use REST, not graphql”, “90% of our endpoints are resource oriented, but we have a couple of endpoints that look rpc-ish; please ignore the latter”… It’s all very stupid.
I had never written an eslint rule until i started having agents pump them out for me and now I've encoded a bunch of important rules as lint rules that will fail CI if violated.
It's like using a compiler that generates semantically different code every time you run it. Basically like compiling a program that's full of UB but "seems to work" most of the time.
> business sees this as productivity gains
Back to LoC/s as a measure of "productivity."
IMO this doesn’t follow from what OP wrote. I personally measure it with a more abstract “how long does it take me to ship something that is useful in production and solving a real problem” and the increase in speed there has been massive for me. But of course I’m not a bigbrain 10x coder that is doing bleeding edge novel stuff like most people here, so gains might be more obvious for me than for others.
But that’s only half of the problem. What about “and how easy it is to maintain long-term”. If you say that maintenance can be done via LLM, I would argue that there is zero guarantees that LLMs are backwards compatible and that the markdown you wrote now will work just as fine in 1,2,3 years
I think the harness and code patching technique starts to matter a lot more once you get outside the trivial range of codebases that fit within the first ~20% of the context window and can otherwise be iterated completely in a single inference pass.
The apply_patch technique that OAI has polished their models on seems to be the best approach for monster scale codebases. Anything based on line ranges and simple find-replace will disintegrate at the edges. You need multiple spatial anchors to deal with nasty things like cshtml files. The prepare/commit behavior is ideal for iterating through ambiguous contexts across many large files and refining anchors.
llms cannot generate anything novel.
Code doesn't need to be novel to be useful. There's a reason why design patterns are a thing in software.
AI is not an abstraction.
https://www.reddit.com/r/math/comments/1tj534d/openais_inter...
So basically 90% of programming in an enterprise environment? lol. Sounds useful to me...
There's plenty of focus on the negative side of the tradeoff. Less so on why we're making it anyway, or why it somehow works out even if "this starts to look like we're all just moving complexity from the more formal and deterministic world of programming languages to the informal and non-deterministic world of natural language".
And the answer to that can be condensed to a one-liner, which I quote after[0]:
sizeof(docs) << sizeof(code)
[0] - https://drensin.medium.com/elephants-goldfish-and-the-new-go... - article may be a bit fluffy here and there, but that one line was a big insight for me.
One major weakness of this study is that they didn’t fully test frontier models for cost reasons, so the specific performance results should be taken with a grain of salt. But the overall conclusion that models degrade when both behavior and architecture must be correct is interesting, and something to keep an eye on.
If you only have functional requirements, then in effect you're doing some form of program synthesis, and RL can optimize that very hard.
If you have a mixture of functional and non-functional requirements, you are basically giving the model an incomplete specification, and it must in some way guess at the user's intent to fill in the blanks. This is also why adding to the prompt examples of the style of code you want (hats off to antirez for this particular tip ;)) is phenomenally powerful.
You could take it a step further and put the example code into source code files...and be like, super comprehensive with your examples ... ;)
To put it in practice: if you point claude/codex to a repository and you ask it to implement feature X using style guide Y, the code will probably work, but you can usually get better results by saying "do it in the style of this file, it was done well there".
The more it has to go on, the more it relies on repetition of what came before. It's also possible that authors start paying much less attention and put less effort into editing later chapters.
Despite the sheer volume on Amazon, LLMs are not at the point of writing well.
For example, if you apply "guardrails" to an image generator of about a year ago, all the people start looking alike. Story generators start using only a few standard names.
That was last year. Is it happening with the frontier models?
I mean, I spend more tokens having them clean up all the places they didn't follow the the plan (if I catch it) or implementing what came out of a 'complete and tested' previous plan where they just stop as soon as all the pathetic new test pass and you discover half of it isn't even there when trying to implement the next thing on top of it.
Though... I have been conducting an experiment, of sorts, where we've been cooking on these fairly complicated projects and I don't ever touch a single line of code, just yell at them a lot, and with suitable amounts of marijuana (they are very frustrating most of the time) it's been going pretty well. I also helps that they need to explain what they're doing to somebody fairly-baked -- maybe not such an HR friendly plan?
I’m not really interested in analysis of the weaknesses of such models because in my experience many weaknesses disappear entirely as models get stronger and reasoning effort is turned up. Especially if you tell them what you want them to do.
Also, it’s not surprising to learn that when more acceptance criteria are added the failure rate increases.
Even the best models have trouble adhering to stuff as mundane as rules for how to style generated code (indent this much, name things with these patterns, etc.). Even the most die-hard AI-first coder will admit to that kind of stuff being not unheard-of. Yet they still delude themselves into thinking that these models will follow a sufficiently detailed spec to the letter, every time.
I've only read the abstract of this one so far but it seems like this paper has zoomed in on programming with greater fidelity and shown a similar phenomenon. But not about long horizon tasks, more like "long style horizons" of larger sets of structural constraints.
[1] https://arxiv.org/abs/2604.15597
Discussion: https://news.ycombinator.com/item?id=48073246
sounds like an oxymoron of a claim.
When designing a system or a component we have ideas that form invariants. Sometimes the invariant is big, like a certain grand architecture, and sometimes it’s small, like the selection of a data structure. Except, eventually, you’ll want to add a feature that clashes with that invariant. At that point there are usually three choices:
- Don’t add the feature. The invariant is a useful simplifying principle and it’s more important than the feature; it will pay dividends in other ways.
- Add the feature inelegantly or inefficiently on top of the invariant. Hey, not every feature has to be elegant or efficient.
- Go back and change the invariant. You’ve just learnt something new that you hadn’t considered and puts things in a new light, and it turns out there’s a better approach.
Often, only one of these is right. Often, at least one of these is very, very wrong, and with bad consequences. Even when they are able to follow constraints, agents are terrible at identifying when the constraints need to change.
[1]: https://medium.com/@vishvananda/i-spent-2-billion-tokens-wri...
FWIW I've noticed this too. I've found that the agents/models have their own style, which is mostly summed up as overly verbose.
Additionally, the models are OK at modularization when given space to "plan" their implementation, but rarely decide that abstracting something would be helpful after the fact (i.e. after many iterations on a greenfield codebase or when being dropped into a legacy codebase).
This often leads to "god files" which, when pointed to by the user/architect, causes the models to correctly critique (humorously when they're the ones that wrote the code in the first place).
What has been you experience? What has your production code looked like in recent months?
If there's a second thing the generative AI tools have shown beyond any doubt it's that many of the more modern (relatively speaking) "best practices" that have always been over-hyped and questionably-evidenced really do tend to produce worse results. LLMs take these methods to their logical conclusions and show us the end result much sooner. You can't just iterate your way to a solution when you don't even know what problem you're trying to solve. If you don't have a clear spec then you don't know what a correct product looks like. You need to invest time in reviewing code properly. If you don't keep the big picture in mind then the big picture becomes a mess.
Maybe one day the LLMs will leave me out of a job but at least I'll feel validated first!
When using Codex/Claude Code with Go code I cannot count the times the agent does some change, runs a build to check for errors, find some and fix them.
https://docs.python.org/3/library/typing.html
"The Python runtime does not enforce function and variable type annotations. They can be used by third party tools such as type checkers, IDEs, linters, etc."
Which third-party enforcement mechanism do you propose become the default?
There are many reasons for this. A big one is that many libraries are only partially typed at best, and dynamic types tend to propagate, weakening the guarantees you get from type checking.
Dynamic idioms in general, including something as common as string-indexed dictionaries, negate type checking. Runtime metaprogramming is the same. All of these things have equivalents in a good statically checked language, but Python doesn't follow those models.
Fundamentally, in Python static typing is an optional analysis layer over a dynamic language, and the consequences of that can't be fully mitigated. The result is a big difference in what types can guarantee.
> Dynamic idioms in general, including something as common as string-indexed dictionaries, negate type checking.
Do you have any proof of this? It hasn't been a problem in TypeScript, and I doubt it's an issue in Python
tasks spanning eight web frameworks
The most incredible combo I've seen lately is progressive enhancement of Razor Pages with javascript. With this arrangement the newest models tend to make a really good call on if something should happen server-side (cshtml) or on the client (js).
For a little complex changes, I always run codex (5.5-high) in planning mode first. I have linked various docs/{ARCHITECTURE,BACKEND-GUIDELINES,NESTJS-DI,..}.md etc. from AGENTS.md so they can quickly discover relevant docs at planning time, only if they are needed. No need to know react specific stuff when it's dealing with a backend problem for example. I typically blindly approve plans made by the agent with a fresh context, because that's as if I had prompted it. Works the best for me.
Using /goal however, it's really just constantly compacting and doing it's thing, of course it gets sloppy. If only there was a state machine that would transform tickets into a Planning Mode Prompt, then use, idk. guardian approvals (somehow a "Product Management Perspective Lens" approving or making changes to the plan) and then letting a less capable or less reasoning agent execute the plan, I think that would work the best.
Aspiration vs. consequence, in other words. An aspiration constraint describes a desired outcome for the system; a consequence constraint maps to a problem already encountered. And the agent ignores the former when faced with the path of least resistance while obeying the latter because it is brief, unambiguous, and precise about preventing that particular failure mode. Which is key rather than the harness in determining survival through session rotation.
These two projects are on GitHub, you may search alexwwang/aristotle and alexwwang/tdd-pipeline to dive into the details or just ask your LLM to scan them to tell you the points you are interested in.
Anyone read whether these tests include any validation loops? What happens if the models get back test failures, for instance? Understanding how many turns to hit full passing behavior suite would also be interesting. Great methodology in the study though.
But they're also superhuman in so many other ways. It's valid to point out limitations, but that doesn't support the conclusion that models are not incredibly powerful and capable of the functional equivalent of reasoning at human or superhuman levels in many scenarios.
In our story, investors are mining intelligence from GPUs, and they truly believe they are one inch from discovering the biggest goldmine in history. But GPUs, unlike a goldmine, cannot be inspected for traces of gold by independent contractors. To keep the hype up, the nihilists in our story dig up cheap gold-looking metals from time to time and tell investors that with a bit of alchemy - agentic workflows, etc. - those metals can be magically turned into gold.
Investors will keep digging until the end of the age, or until they run out of money.
A framework would use static code checking tools to force an architecture on to LLMs instead of trying to do so in markdown.
I don't know exactly what it will look like but for example I could imagine a Java Framework where the LLM could only create subclasses of certain classes.
Implementation phases very often go through 5-10 review and fix rounds to actually get the implementation to match the spec. It takes longer but that's what's necessary to get actually good results on long horizon tasks with detailed requirements. I'll be open sourcing it fully soon.
I would agree too that as the codebase grows the LLM struggles more and more with generating code. It is probably misaligned incentives, it wants to complete the isolated task without too much context consumed, at the POC it can consume most of the app, by about 30K lines of code it is quite complex code base to navigate.
Time to start writing linting tools that check the architecture and spoon feed the LLM what exactly it's doing wrong.
I reckon something like this would be good for every project out there: https://www.archunit.org/getting-started
They expand a bit more on the reasoning behind it: https://www.archunit.org/motivation
(I also wrote a simple linter for architecture/code checks that aren't well encapsulated by ones that just focus on individual files, that uses Go + goja to write rules in ECMAScript and parallelize the read only ones and also allow ones that change files as necessary, in addition to something like Ruff / Oxlint / Oxfmt / whatever is present in each stack; though it's is still in development and not as good of a focused example as ArchUnit is)
If we write software specification docs, bother describing how it evolves with ADRs, enforce code style automatically and require certain test coverage automatically (or at least should), why couldn't we go a step further, formalize those specs and ensure that any new code is also up to snuff? I don't think that's any more of a job for an LLM, than telling it how it should format code is. Also, I'm in the camp that believes that at least many of your ORM mappings and similar stuff should be the output of codegen, since you've already gone through the trouble of describing the schema/migrations to get there.
I don't think this would be only good for LLMs, though - I've seen projects that have like 3 different audit systems built in, not because of some fancy business requirement, but rather cause the devs either didn't know about the previous one(s) or just didn't feel like following what should have been the pre-established conventions, even when there were docs in place (nobody read those).
I have exactly the inverse findings on my end. The bigger and more legacy the codebase, the more accurate the patches become.
The harness itself seems to be the most important part. I use a recursive loop that primes the root context based on the user prompt each time. My agent will often make over 100 tool calls to sql and git before it finally decides to apply a patch. If I was greenfield, there would be nothing to query or constrain against.
I usually find I can achieve 90% of the outcome I'm trying to achieve. I use sonnet for planning, qwen for coding, sonnet for review.
gpt 5.2 is the strongest model they tested, a nearly 6 month old model.
Traditional research can not keep up.