It also has no concept of what it means for the choice of token to be an “error” or not, or what a “correct” answer would be.
If I put a weight on one side of a die, and I roll it, the die is not more confident that it will land on that side than it would be otherwise, because dice do not have the ability to be confident. Asserting otherwise shows a fundamental misunderstanding of what a die is.
The same is true for LLMs.
But there are algorithms with stopping conditions (Newton-Raphson, gradient descent), and you could say that an answer is "uncertain" if it hasn't run long enough to come up with a good enough answer yet.
Personally, until recently I can only recall people saying things along the lines of “applying the model indicates that we can state this fact about the data with this much confidence”, never “the model has this much confidence” in some truth statement, especially one independent of its training data.
Yes, LLMs have a pretty good idea of the uncertainty and truth of their predictions internally. https://news.ycombinator.com/item?id=41418486
- An LLM knowing when it is lying is not the same thing as its internal state being able to “reveal the truthfulness of statements”. The LLM does not know when it is lying, because LLMs do not know things.
- It is incapable of lying, because lying requires possessing intent to lie. Stating untrue things is not the same as lying.
- As the paper states shortly afterwards, what it actually shows is “given a set of test sentences, of which half are true and half false, our trained classifier achieves an average of 71% to 83% accuracy”. That’s not the same thing as it being able to “reveal the truthfulness of statements”.
No intellectually honest person would claim that this finding means an LLM “knows when it is lying”.
You keep saying the same nonsense over and over again. A LLM does not know things so... What kind of argument is that ? You're working backwards from a conclusion that is nothing but your own erroneous convictions on what a "statistical model" is and are undertaking a whole lot of mental gymnastics to stay there.
There are a lot of papers there that all try to approach this in different ways. You should read them and try to make an honest argument and that doesn't involve "This doesn't count because - claim that is in no way empirically or theoretically validated."
