undefined | Better HN

0 pointsxienze1y ago0 comments

I’ve had multiple occasions where I’ve asked an LLM how to do <whatever> in Java and it’ll very confidently answer to use <some class in some package that doesn’t exist>. It would be far more helpful to me to receive an answer like “I don’t think there’s a third party library that does this, you’ll have to write it yourself” than to waste my time telling me a lie. If anything, calling these outputs “hallucinations” is a very polite way of saying that the LLM is bullshitting the user.

0 comments

Gormo1y ago

Of course the LLM is bullshitting the user. That's precisely its purpose: LLMs are tools that generate comprehensible sounding language based on probability models that describe what words/tokens tend to be found in proximity to each other. An LLM doesn't actually know anything by reference to verifiable, external facts.

Sure, LLMs can be used as fancy search engines that index documents and then answer questions by referring to them, but even there, the probabilistic nature of the underlying model can still result in mistakes.

mike_hearn1y ago

Models do know things. Facts are encoded in their parameters. Look at the some of the interpretability research to see that. They aren't just Markov chains.

Gormo1y ago

Nope. They don't know any specific facts. The training data produces a probability matrix that reflects what words are likely to be found in relation other words, allowing it to generate novel combinations of words that are coherent and understandable. But there is no mechanism involved for determining whether those novel expressions are actually factual representations of reality.

1 more reply

xienzeOP1y ago

Yeah I get that, but at the same time we have AI hype men talking out of both sides of their mouth:

> This model is revolutionary, it knows everything, can answer anything with perfect accuracy!

“It’s fed me bullshit numerous times”

> OF COURSE it’s bullshitting you, don’t you know how LLMs work?

Like how am I supposed to take any of this tech seriously when the LLM is always answering questions as if it had the utmost confidence in what it is spitting out?

dumpsterdiver1y ago

Hilariously, that really does basically define “bullshitting”.

Al-Khwarizmi1y ago

Bullshit in the Frankfurtian sense.

There is a recent paper that explains it: https://link.springer.com/article/10.1007/s10676-024-09775-5

lmm1y ago

The LLM is always bullshitting the user. It's just sometimes the things it talks about happen to be real and sometimes they don't.

tengbretson1y ago

LLMs don't know things, they just string together responses that are a best fit for what follows from their prompt.

I suspect its so hard to get them to say "I don't know" because if they were biased towards responding that way then I would assume thats almost all they would ever say, since "I don't know" is an appropriate answer to every question imaginable.

JKCalhoun1y ago

I get that, but since it is all probabilities, you might imagine even the LLM knows when it is skating on thin ice.

If I'm beginning with "Once / upon / a" I think the data will show a very high confidence in the word to follow with. So too I would imagine it would know when the trail of breadcrumbs it has been following is of the trashier and low probability kind.

So just tell me. (Or perhaps speak to me and when your confidence is low you can drift into vocal fry territory.)

Gormo1y ago

Maybe just having a confidence weight assigned to each sentence the LLM generates, reflected in tooltips or text coloring, would be a big improvement.

namaria1y ago

> the LLM knows

I don't think you get it.

2 more replies

j / k navigate · click thread line to collapse

0 comments

Gormo1y ago

mike_hearn1y ago

Models do know things. Facts are encoded in their parameters. Look at the some of the interpretability research to see that. They aren't just Markov chains.

Gormo1y ago

1 more reply

xienzeOP1y ago

Yeah I get that, but at the same time we have AI hype men talking out of both sides of their mouth:

> This model is revolutionary, it knows everything, can answer anything with perfect accuracy!

“It’s fed me bullshit numerous times”

> OF COURSE it’s bullshitting you, don’t you know how LLMs work?

Like how am I supposed to take any of this tech seriously when the LLM is always answering questions as if it had the utmost confidence in what it is spitting out?

dumpsterdiver1y ago

Hilariously, that really does basically define “bullshitting”.

Al-Khwarizmi1y ago

Bullshit in the Frankfurtian sense.

There is a recent paper that explains it: https://link.springer.com/article/10.1007/s10676-024-09775-5

lmm1y ago

The LLM is always bullshitting the user. It's just sometimes the things it talks about happen to be real and sometimes they don't.

tengbretson1y ago

LLMs don't know things, they just string together responses that are a best fit for what follows from their prompt.

JKCalhoun1y ago

I get that, but since it is all probabilities, you might imagine even the LLM knows when it is skating on thin ice.

So just tell me. (Or perhaps speak to me and when your confidence is low you can drift into vocal fry territory.)

Gormo1y ago

Maybe just having a confidence weight assigned to each sentence the LLM generates, reflected in tooltips or text coloring, would be a big improvement.

namaria1y ago

> the LLM knows

I don't think you get it.

2 more replies

j / k navigate · click thread line to collapse