undefined | Better HN

0 pointscodedokode7mo ago0 comments

Yes but in matrix multiplication there are O(N²) numbers and O(N³) multiplications, so it might be possible that you are bounded by compute speed.

0 comments

both are equally important. compute for prefill and mem bandwidth for generation

j / k navigate · click thread line to collapse

both are equally important. compute for prefill and mem bandwidth for generation

j / k navigate · click thread line to collapse