undefined | Better HN

0 pointsfnordpiglet22d ago0 comments

I think the key stats are this (for m5 max)

M5 128GB RAM with 614GB/s memory transfer

This is a huge step over M4 32GB 153GB/s memory transfer

For local LLM this make it a replacement for a DGX Spark, which offers a third of the transfer speed and is not something you toss in your backpack as your laptop. It’s practically useful for a lot of local use cases and that I think is the 4x factor (memory xfer) - but the 128Gb unified headroom tremendously improves the models you can run and training you can do.

0 comments

sroussey22d ago

You are comparing a M5 Max to a base M4.

The M4 Max has 546 GB/s compared to 614GB/s for the M5 Max. Which is like 12% faster not 4x.

zmmmmm22d ago

What is truly amazing is the M1 Max is 400GB/s. 5 years later and we still only hit 1.5x on memory bandwidth. It's quite fascinating how high Apple spec'd it back then with apparently little foreknowledge of how important memory bandwidth would become, and then conversely how little they've managed to improve it now when it's so obvious how important it is.

ColonelPhantom22d ago

The reason for that is that most memory bandwidth bumps come with new memory generations. For example an early DDR4 platform (e.g. Intel Skylake/Core iX-6000) and a late one (e.g. AMD Zen3/Ryzen 5000) only differ by 1.5x as well, typically.

The same trend is visible in GPUs: for example, my RTX 2070 (GDDR6) has the same memory bandwidth as a 3070 and only a little bit less than a 4070 (GDDR6X). However, a 5070 does get significantly more bandwidth due to the jump to GDDR7. Lower-end cards like the 4060 even stuck to GDDR6, which gave them a bandwidth deficit compared to a 3060 due to the narrower memory buses on the 40 series.

1 more reply

j / k navigate · click thread line to collapse