undefined | Better HN

0 pointsdvfjsdhgfv8mo ago0 comments

As you say, we will never know, but this article[0] claims:

> The cost of the compute to train models alone ($3 billion) obliterates the entirety of its subscription revenue, and the compute from running models ($2 billion) takes the rest, and then some. It doesn’t just cost more to run OpenAI than it makes — it costs the company a billion dollars more than the entirety of its revenue to run the software it sells before any other costs.

[0] https://www.lesswrong.com/posts/CCQsQnCMWhJcCFY9x/openai-los...

0 comments

matwood8mo ago

CapEx vs. OpEx.

If they stop training today what happens? Does training always have to be at these same levels or will it level off? Is training fixed? IE, you can add 10x the subs and training costs stay static.

IMO, there is a great business in there, but the market will likely shrink to ~2 players. ChatGPT has a huge lead and is already Kleenex/Google of the LLMs. I think the battle is really for second place and that is likely dictated by who runs out of runway first. I would say that Google has the inside track, but they are so bad at product they may fumble. Makes me wonder sometimes how Google ever became a product and verb.

marcosdumay8mo ago

That paragraph is quite clear.

OpEx is larger than revenue. CapEx is also larger than the total revenue on the lifetime of a model.

ghc8mo ago

Obviously you don't need to train new models to operate existing ones.

I think I trust the semianalysis estimate ($250M) more than this estimate ($2B), but who knows? I do see my revenue estimate was for this year, though. However, $4B revenue on $250M COGS...is still staggeringly good. No wonder amazon, google, and Microsoft are tripping over themselves to offer these models for a fee.

singron8mo ago

You need to train new models to advance the knowledge cutoff. You don't necessarily need to R&D new architectures, and maybe you can infuse a model with new knowledge without completely training from scratch, but if you do nothing the model will become obsolete.

Also the semianalysis estimate is from Feb 2023, which is before the release of gpt4, and it assumes 13 million DAU. ChatGPT has 800 million WAU, so that's somewhere between 115 million and 800 million DAU. E.g. if we prorate the cogs estimate for 200 DAU, then that's 15x higher or $3.75B.

ghc8mo ago

> You need to train new models to advance the knowledge cutoff

That's a great point, but I think it's less important now with MCP and RAG. If VC money dried up and the bubble burst, we'd still have broadly useful models that wouldn't be obsolete for years. Releasing a new model every year might be a lot cheaper if a company converts GPU opex to capex and accepts a long training time.

> Also the semianalysis estimate is from Feb 2023,

Oh! I missed the date. You're right, that's a lot more expensive. On the other hand, inference has likely gotten a lot cheaper (in terms of GPU TOPS) too. Still, I think there's a profitable business model there if VC funding dries up and most of the model companies collapse.

dvfjsdhgfvOP8mo ago

> Obviously you don't need to train new models to operate existing ones.

For a few months, maybe. Then they become obsolete and, in some cases like coding, useless.

hamburga8mo ago

But assuming no new models are trained, this competitive effect drives down the profit margin on the current SOTA models to zero.

ghc8mo ago

Even if the profit margin is driven to zero, that does not mean competitors will cease to offer the models. It just means the models will be bundled with other services. Case in point: Subversion & Git drove VCS margin to zero (remember BitKeeper?), but Bitbucket and Github wound up becoming good businesses. I think Claude Code might be the start of how companies evolve here.

j / k navigate · click thread line to collapse

0 comments

matwood8mo ago

CapEx vs. OpEx.

If they stop training today what happens? Does training always have to be at these same levels or will it level off? Is training fixed? IE, you can add 10x the subs and training costs stay static.

marcosdumay8mo ago

That paragraph is quite clear.

OpEx is larger than revenue. CapEx is also larger than the total revenue on the lifetime of a model.

ghc8mo ago

Obviously you don't need to train new models to operate existing ones.

singron8mo ago

ghc8mo ago

> You need to train new models to advance the knowledge cutoff

> Also the semianalysis estimate is from Feb 2023,

dvfjsdhgfvOP8mo ago

> Obviously you don't need to train new models to operate existing ones.

For a few months, maybe. Then they become obsolete and, in some cases like coding, useless.

hamburga8mo ago

But assuming no new models are trained, this competitive effect drives down the profit margin on the current SOTA models to zero.

ghc8mo ago

j / k navigate · click thread line to collapse