Inference isn’t that expensive. A single junior dev costs orders of magnitude more than the amount of inference I use. Companies in growth mode don’t have to make money, it’s a land grab right now. But the expense is largely in the R and D. You can build a rig to run full models for 10-20k right? That’s only a month or two of a junior dev’s time, and after that it’s just electricity. And you could have dozens of devs using the same rig as long as they could timeshare. I don’t see where the economics wouldn’t work, it’s just there’s no use in investing in the hardware until we know where AI is going.
Yeah, you can build a rig to run full models for 10-20k... That's a big reason OpenAI might not make it. The whole article is about LLMs not being a viable business.