Not saying adding few novel ideas (perhaps working world models) to the current AI toolbox won't make a breakthrough, but LLMs have their limits.
https://en.wikipedia.org/wiki/Ninety%E2%80%93ninety_rule
Except that the either side of it is immensely cheaper now.
How long before those lines cross? Intuitively it feels like we have about 2-3 years before claude is better at writing code than most - or all - humans.
I don't see it in practice though.
The fundamental problem hasn't changed: these things are not reasoning. They aren't problem solving.
They're pattern matching. That gives the illusion of usefulness for coding when your problem is very similar to others, but falls apart as soon as you need any sort of depth or novelty.
I haven't seen any research or theories on how to address this fundamental limitation.
The pattern matching thing turns out to be very useful for many classes of problems, such as translating speech to a structured JSON format, or OCR, etc... but isn't particularly useful for reasoning problems like math or coding (non-trivial problems, of course).
I'm pretty excited about the applications for AI overall and it's potential to reduce human drudgery across many fields, I just think generating code in response to prompts is a poor choice of a LLM application.
And, pray tell, how people are going to come up with such design?
The other day I tested an AI by giving it a folder of images, each named to describe the content/use/proportions (e.g., drone-overview-hero-landscape.jpg), told it the site it was redesigning, and it did a very serviceable job that would match at least a cheap designer. On the first run, in a few seconds and with a very basic prompt. Obviously with a different AI, it could understand the image contents and skip that step easily enough.
It's kind of telling that the number of apps on Apple's app store has been decreasing in recent years. Same thing on the Android store too. Where are the successful insta-apps? I really don't believe it's happening.
https://www.appbrain.com/stats/number-of-android-apps
I've recently tried using all of the popular LLMs to generate DSP code in C++ and it's utterly terrible at it, to the point that it almost never even makes it through compilation and linking.
Can you show me the library of apps you've launched in the last few years? Surely you've made at least a few million in revenue with the ease with which you are able to launch products.
They wouldn’t even know where to begin!
Even if all sandboxing is done right, programs will be depended on to store data correctly and to show correct outputs.
I think exceptional work, AI tools or not, still takes exceptional people with experience and skill. But I do feel like a certain level of access to technology has been unlocked for people smart enough, but without the time or tools to dive into the real industry's tools (figma, code, data tools etc).
I think the idea that LLM's will usher in some new era where everyone and their mom are building software is a fantasy.
