undefined | Better HN

0 pointsthrow3108225d ago0 comments

Interesting. So you expect it to "not think" and simply produce a value corresponding to "it's the same to me", knowing that it will be translated into an actual random value.

Instead, exactly as a person would do, it does think of a specific number that feels random in that particular moment.

0 comments

pmontra5d ago

If I care a little bit about that random number I might reach for my phone and look at the digits of the seconds of the current time. It's 31 now. Not appropriate for multiple lookups.

throw310822OP5d ago

Yes, there is probably some variable context in every chat (like date and time). Could work as a good seed but I guess you should ask the LLM to really make an effort to produce a seriously random number. (Actually I've just tried, even if you ask it to make an effort, the number will be always the same).

gzread5d ago

If "a random number is" is followed by all digits equally in the training dataset, then emitting a uniform distribution should minimise cross-entropy, right?

throw310822OP4d ago

Er, should it? Even of you trained an LLM exactly over this type of question, if the sequence to predict in the training data is really random, then any output is equally wrong. Even if the output is a fixed "1234" or "0000". There is no signal to train on, not even one that favours an equal distribution.

On the other hand, LLMs show they know very well what a random number is and the fact it just shouldn't look like anything in particular, so they strive to come up with a number that doesn't look like anything in particular. Which happens to be always the same number given the starting conditions.

j / k navigate · click thread line to collapse