undefined | Better HN

0 pointsvrm7mo ago0 comments

a 6:1 parameter ratio is too small for specdec to have that much of an effect. You'd really want to see 10:1 or even more for this to start to matter

0 comments

lhl7mo ago

You're right on ratios, but actually the ratio is much worse than 6:1 since they are MoEs. The 20B has 3.6B active, and the 120B has only 5.1B active, only about 40% more!

j / k navigate · click thread line to collapse

0 pointsvrm7mo ago0 comments

a 6:1 parameter ratio is too small for specdec to have that much of an effect. You'd really want to see 10:1 or even more for this to start to matter

0 comments

lhl7mo ago

You're right on ratios, but actually the ratio is much worse than 6:1 since they are MoEs. The 20B has 3.6B active, and the 120B has only 5.1B active, only about 40% more!

j / k navigate · click thread line to collapse