undefined | Better HN

0 pointsBirAdam3d ago0 comments

The speed on a constrained device isn't entirely the point. Two years ago, LLMs failed at answering coherently. Now...

You're absolutely right. Now, LLMs are too slow to be useful on handheld devices, and the future of LLMs is brighter than ever.

LLMs can be useful, but quite often the responses are about as painful as LinkedIn posts. Will they get better? Maybe. Will they get worse? Maybe.

0 comments

vntok3d ago

> Will they get better? Maybe. Will they get worse? Maybe.

I find it hard to understand your uncertainty; how could they not keep getting even better when we've been seeing qualitative improvements literally every second week for months on end? These improvements being eminently public and applied across multiple relevant dimensions: raw inference speed (https://github.com/ggml-org/llama.cpp/releases), external-facing capabilities (https://github.com/open-webui/open-webui/releases) and performance against established benchmarks (https://unsloth.ai/docs/models/qwen3.5/gguf-benchmarks)

BirAdamOP2d ago

There are many metrics for “better” and “worse”. It is entirely possible for an AI system to be better in the sense of hallucination while also being of less utility. An arrogant prick who’s always correct isn’t always a good person to have on your team, right?

j / k navigate · click thread line to collapse