undefined | Better HN

0 pointsnl4d ago0 comments

I always wonder what the world is going to be like when these are 2 orders of magnitude faster.

Something like https://chatjimmy.ai/ will do 14,000 tokens per second, and it's a completely different experience to what we see now. It's more like a page load than a conversation.

0 comments

jjbinx0074d ago

That's really impressive from a speed point of view but it hallucinates like it's on drugs.

I'd much rather trade speed for accuracy.

gunalx4d ago

Its llama 3.1 8b. So not a particular good model, mostly as a tech demo of the speed possible.

j / k navigate · click thread line to collapse

0 comments

jjbinx0074d ago

That's really impressive from a speed point of view but it hallucinates like it's on drugs.

I'd much rather trade speed for accuracy.

gunalx4d ago

Its llama 3.1 8b. So not a particular good model, mostly as a tech demo of the speed possible.

j / k navigate · click thread line to collapse