undefined | Better HN

0 pointsadastra223mo ago0 comments

Again, memory bandwidth is pretty much all that matters here. During inference or training the CUDA cores of retail GPUs are like 15% utilized.

0 comments

my1233mo ago

Not for prompt processing. Current Macs are really not great at long contexts

j / k navigate · click thread line to collapse

0 pointsadastra223mo ago0 comments

Again, memory bandwidth is pretty much all that matters here. During inference or training the CUDA cores of retail GPUs are like 15% utilized.

my1233mo ago

Not for prompt processing. Current Macs are really not great at long contexts

j / k navigate · click thread line to collapse