undefined | Better HN

0 pointsbenedictevans7d ago0 comments

Yes, I wrote about Rock’s Law too, but we don’t know that this is how these models will develop

0 comments

Evidence point to the same type of scaling law. Compute for a training run grows 4-5x every year.[0] I'm sure this will slow down but the premise remains that weaker competitors will not be able to maintain this pace. We already see labs like Cohere, Mistral, Inflection AI, Adept, Character.ai, and others bow out of the frontier race. I'm also skeptical that Meta, xAI can catch up. Even Google has trouble keeping up.

Even if this isn't true, comparing telecom bits to tokens is wrong. Bits are the same no matter what telecom transfers them. Tokens are not all the same. The quality varies.

We're already seeing a massive divide between frontier models and lesser models in growth rates. Anthropic is adding $10b - $15b every month in ARR. This figure likely dwarfs open source labs. This is all because its models are maybe 10-15% better.

The cost to inference a 1T param frontier model is the same as a 1T param open source model. Therefore, if the frontier model is even 10-15% better, it will gobble up the market over time.

Lastly, even though Claude Code and Codex are the biggest revenue drivers for Anthropic and OpenAI today, I don't believe this will be true 2-5 year from now. I believe selling their tokens via API will be their biggest. The sum of applications in the world will dwarf coding in market size. For example, biotech, finance, physics, engineering, robotics, sensor data, etc. This is why I think OpenAI and Anthropic are becoming more like iOS and Android than AT&T and Verizon. Applications will build on top of OpenAI and Anthropic just like iOS and Android.

[0]https://epoch.ai/blog/training-compute-of-frontier-ai-models...

fittingopposite6d ago

How about the externalized intelligence around the model weights (skills, tools, harness, memory etc)? If the model weights are sufficiently intelligent, the focus might move to the external layers.

svnt6d ago

I agree with much of what you’ve written but think you are missing the correct alignment of the mobile data timeline — mobile data had standards because it was forced to. It was forced to early because it was not a fundamental innovation, telecom itself was the fundamental innovation, mobile was a constraint relaxation. Intelligence might be forced to have standards as well, we will see what form the regulations take when prices reflect costs and healthy margins and become existential threats for many businesses.

aurareturn6d ago

Intelligence can’t be standardized.

The reason mobile data had to standardize is because it’s a network and a network must have protocols. It’s useless without them.

svnt6d ago

I agree with that as a premise, but again it seems to me you are selectively jumping way into the end game. There were early networks that did not standardize, and these nonstandard networks had advantages, and some of those advantages were sacrificed in market-driven standardization.

Intelligence must have interfaces, and those can be standardized. Businesses will try to remain provider agnostic, which will also drive standardization via standard sales and marketing methods.

Separately, we are doing our best to standardize performances on benchmarks.

I don’t disagree that right now transport of standardized mobile data vs emulation of human intelligence is qualitatively different, but perhaps primarily because it is early in development, and our vantage point this time is relatively from within the network, instead of outside it.

menaerus6d ago

You lay out some good arguments but I agree with both: the models relative to few years back really did become the commodity because today you could take the non-frontier model, maybe self-host it or pay the much less price per M tokens to get the performance of a ~2-year old frontier model. At the same time I do think that we are getting into the monopoly/duopoly/tripoly with the frontier models for all the reasons you already mentioned, and this scares me a little bit.

aurareturn6d ago

Lower intelligence LLMs can be a commodity, yes. But these won't make much money, if at all. At the end of the day, it costs the same to inference a 1T frontier model and a 1T free model.

OpenAI and Anthropic don't compete in the LLM commodity market. Hence, I had a problem with slide 22.

j / k navigate · click thread line to collapse

0 comments

aurareturn6d ago

Even if this isn't true, comparing telecom bits to tokens is wrong. Bits are the same no matter what telecom transfers them. Tokens are not all the same. The quality varies.

The cost to inference a 1T param frontier model is the same as a 1T param open source model. Therefore, if the frontier model is even 10-15% better, it will gobble up the market over time.

[0]https://epoch.ai/blog/training-compute-of-frontier-ai-models...

fittingopposite6d ago

How about the externalized intelligence around the model weights (skills, tools, harness, memory etc)? If the model weights are sufficiently intelligent, the focus might move to the external layers.

svnt6d ago

aurareturn6d ago

Intelligence can’t be standardized.

The reason mobile data had to standardize is because it’s a network and a network must have protocols. It’s useless without them.

svnt6d ago

Intelligence must have interfaces, and those can be standardized. Businesses will try to remain provider agnostic, which will also drive standardization via standard sales and marketing methods.

Separately, we are doing our best to standardize performances on benchmarks.

menaerus6d ago

aurareturn6d ago

Lower intelligence LLMs can be a commodity, yes. But these won't make much money, if at all. At the end of the day, it costs the same to inference a 1T frontier model and a 1T free model.

OpenAI and Anthropic don't compete in the LLM commodity market. Hence, I had a problem with slide 22.

j / k navigate · click thread line to collapse