undefined | Better HN

0 pointsACCount371d ago0 comments

Full distributions are a fucking pain to save - at this point just save the hidden states. But there are lossy compression tricks there.

0 comments

rao-v18h ago

To the previous poster's point, soft distributions are useful, even saving the top 10 logits is significantly more training signal than just the final token.

j / k navigate · click thread line to collapse

0 comments

rao-v18h ago

To the previous poster's point, soft distributions are useful, even saving the top 10 logits is significantly more training signal than just the final token.

j / k navigate · click thread line to collapse