undefined | Better HN

0 pointsbrookst1y ago0 comments

Or if you want a large model but don’t need high performance, get a Mac with 128GB UMA.

0 comments

How many tokens/s would you get in such a setup?

This Reddit thread says an m3 max 128GB gets 23 tokens/sec with deepseek r1 32B, and 4 tokens / sec with 70b: https://www.reddit.com/r/LocalLLaMA/comments/1i69dhz/deepsee...

j / k navigate · click thread line to collapse

0 pointsbrookst1y ago0 comments

Or if you want a large model but don’t need high performance, get a Mac with 128GB UMA.

How many tokens/s would you get in such a setup?

brookstOP1y ago

This Reddit thread says an m3 max 128GB gets 23 tokens/sec with deepseek r1 32B, and 4 tokens / sec with 70b: https://www.reddit.com/r/LocalLLaMA/comments/1i69dhz/deepsee...

j / k navigate · click thread line to collapse