It might (currently) be most apt to characterize these occurrences as shifts out of the training data distribution
So far I'm coming around to the growing use of "slop", originally meant as an alternative to "spam" and to imply spam-like intent, but the great thing about this word choice is that the closest anthropomorphic connotation is to "pig feeding". Pigs can be highly intelligent, of course, but that's not the first image one has when thinking of a pig at a slop trough.
From Merriam webster: to fill in gaps in memory by fabrication
That's one definition of bullshitting, but not the one being used here. If someone says "I think you're bullshitting me" then yes, you're being accused of consciously seeking personal gain. But if someone says "we were standing around bullshitting" then no, it refers to killing time with mindless communication, which is a quite good analogy for LLM output.
The LLM does it because it's programmed to, and the human does it for some other self gain reason, but both the process and the results are very similar.
That's my second definition! Sorry if I wasn't clear. My first definition (which aligns with the comment I had originally quoted) is that the speaker is aware that they're saying false things, and therefore has intent to deceive, typically for personal gain (they are bullshitting another person). My second definition is that the speaker has no regard for whether what they're saying is true or false (they are bullshitting with another person).
An LLM does not bullshit you, it bullshits with you. It's fluff, not a bluff.
"hallucinate" does the same thing, fwiw
