undefined | Better HN

0 pointsyencabulator2d ago0 comments

I think we're using different meaning of "behaves like". I meant "has tokens/sec performance comparable to".

0 comments

I'm using model performance because inference is definitely not comparable to a 17B model when you're streaming model weights on and off disk storage.

j / k navigate · click thread line to collapse

0 pointsyencabulator2d ago0 comments

I think we're using different meaning of "behaves like". I meant "has tokens/sec performance comparable to".

0 comments

BoorishBears1d ago

I'm using model performance because inference is definitely not comparable to a 17B model when you're streaming model weights on and off disk storage.

j / k navigate · click thread line to collapse