undefined | Better HN

0 pointsecommerceguy3h ago0 comments

My 2 cents is in 3 years the inference products will be a commodity, extremely competitive with diminishing returns, seeing that the open weight models are getting so good and nearing par with sota.

I feel 90% of sota for 10% of cost / compute is good enough for 80% of workloads.

Question is, do we need all of this hardware? Are build outs going to get canceled. 4300 new data centers seems excessive. I personally haven't experienced any service disruptions...

Does Microsoft still have 1 million gpu's in storage? Is that what I heard earlier this year?

0 comments

dghlsakjg3h ago

For many purposes they already are a commodity. Openrouter will automatically route your requests to cheaper providers of the same model on the fly. Many of the hosts for open source models are basically undifferentiated. It’s a pure price dominated market except at the very top edge. Even there, we are seeing very little lock in. If OpenAI released an Opus beater at half the cost, even large businesses could switch providers almost instantly.

gip2h ago

Yes and this is in line with the idea that 99%+ of the value created by AI will be captured by the broader economy, not OpenAI and friends.

That being said, even local AI will still be displacing humans fast and it's not clear new jobs will be created fast enough. Regulations and policies will be needed imo.