undefined | Better HN

0 pointsplagiarist1y ago0 comments

Sortof a letdown that after 24 32Gb RAM sticks you only get 6-8 tokens per second.

0 comments

But a token is not just a character.

"hello how are you today?" - 7 tokens.

And this is so much better than I could have imagined in a very short span of time.

And only get to use 20k context length before it OOMs

j / k navigate · click thread line to collapse

But a token is not just a character.

"hello how are you today?" - 7 tokens.

And this is so much better than I could have imagined in a very short span of time.

And only get to use 20k context length before it OOMs

j / k navigate · click thread line to collapse