undefined | Better HN

0 pointskadushka10mo ago0 comments

How in the world is this easy and cheap? Are you planning to run this LLM inside the vending machine? Or are you planning to send those prompts to a remote LLM somewhere?

0 comments

geysersam10mo ago

The premise here is that the model runs fast and cheap. With the current state of the technology running a vending machine using an LLM is of course absurd. The point is that accuracy is not the only dimension that brings qualitative change to the kind of applications that LLMs are useful for.

kadushkaOP10mo ago

Running a vending machine using an LLM is absurd not because we can't run LLMs fast or cheap enough - it's because LLMs are not reliable, and we don't know yet how to make them more reliable. Our best LLM - o3 - doubled the previous model (o1) hallucination rate. OpenAI says it hallucinated a wrong answer 33% of the time in benchmarks. Do you want a vending machine that screws up 33% of the time?

Today, the accuracy of LLMs is by far a bigger concern (and a harder problem to solve) than its speed. If someone releases a model which is 10x slower than o3, but is 20% better in terms of accuracy, reliability, or some other metric of its output quality, I'd switch to it in a heartbeat (and I'd be ready to pay more for it). I can't wait until o3-pro is released.

geysersam10mo ago

Do you seriously think a typical contemporary LLM would screw up 33% of vending machine orders?

I don't know what benchmark you're looking at but I'm sure the questions in it were more complicated than the logic inside a vending machine.

Why don't you just try it out? It's easy to simulate, just tell the bot about the task and explain to it what actions to perform in different situations, then provide some user input and see if it works or not.

K0balt10mo ago

You could run a 3B model on 200 dollars worth of hardware and it would do just fine, 100 percent of the time, most of the time. I could definitely see someone talking it out of a free coke now and then though.

With vending machines costing 2-5k, it’s not out of the question, but it’s hard to imagine the business case for it. Maybe the tantalizing possibility of getting a free soda would attract traffic and result in additional sales from frustrated grifters? Idk.

j / k navigate · click thread line to collapse

0 comments

geysersam10mo ago

kadushkaOP10mo ago

geysersam10mo ago

Do you seriously think a typical contemporary LLM would screw up 33% of vending machine orders?

I don't know what benchmark you're looking at but I'm sure the questions in it were more complicated than the logic inside a vending machine.

K0balt10mo ago

j / k navigate · click thread line to collapse