So we'll find out if this model is real or not by 2-3 months. My guess is that it'll turn out to be another flop like O1. They needed to release something big because they are momentum based and their ability to raise funding is contingent on their AGI claims.
We may have progressed from a 99%-accurate chatbot to one that's 99.9%-accurate, and you'd have a hard time telling them apart in normal real world (dumb) applications. A paradigm shift is needed from the current chatbot interface to a long-lived stream of consciousness model (e.g. a brain that constantly reads input and produces thoughts at 10ms refresh rate; remembers events for years and keep the context window from exploding; paired with a cerebellum to drive robot motors, at even higher refresh rates.)
As long as we're stuck at chatbots, LLM's impact on the real world will be very limited, regardless of how intelligent they become.
Now they just have to make it cheap.
Tell me, what has this industry been good at since its birth? Driving down the cost of compute and making things more efficient.
Are you seriously going to assume that won’t happen here?
Like they've been making it all this time? Cheaper and cheaper? Less data, less compute, fewer parameters, but the same, or improved performance? Not what we can observe.
>> Tell me, what has this industry been good at since its birth? Driving down the cost of compute and making things more efficient.
No, actually the cheaper compute gets the more of it they need to use or their progress stalls.
No they haven't, these results do not generalize, as mentioned in the article:
"Furthermore, early data points suggest that the upcoming ARC-AGI-2 benchmark will still pose a significant challenge to o3, potentially reducing its score to under 30% even at high compute"
Meaning, they haven't solved AGI, and the task itself do not represent programming well, these model do not perform that well on engineering benchmarks.
This type of compute will be cheaper than Claude 3.5 within 2 years.
It's kinda nuts. Give these models tools to navigate and build on the internet and they'll be building companies and selling services.
Significantly better at what? A benchmark? That isn't necessarily progress. Many report preferring gpt-4 to the newer o1 models with hidden text. Hidden text makes the model more reliable, but more reliable is bad if it is reliably wrong at something since then you can't ask it over and over to find what you want.
I don't feel it is significantly smarter, it is more like having the same dumb person spend more thinking than the model getting smarter.
