undefined | Better HN

0 pointsbrcmthrowaway4d ago0 comments

Prefill speed and 27B number?

0 comments

Prefill is around ~600 t/s.

I don't remember what the 27B was, I tried a 27B with different quantization at some point for that one, but I settled on the 31B.

j / k navigate · click thread line to collapse

Prefill is around ~600 t/s.

I don't remember what the 27B was, I tried a 27B with different quantization at some point for that one, but I settled on the 31B.

j / k navigate · click thread line to collapse