undefined | Better HN

0 pointsminimaxir9mo ago0 comments

8 GB of RAM with local LLMs in general is iffy: a 8-bit quantized Qwen3-4B is 4.2GB on disk and likely more in memory. 16 GB is usually the minimum to be able to run decent models without compromising on heavy quantization.

0 comments

hnuser1234569mo ago

But 8GB of Apple RAM is 16GB of normal RAM.

https://www.pcgamer.com/apple-vp-says-8gb-ram-on-a-macbook-p...

minimaxirOP9mo ago

Interestingly it was AI (Apple Intelligence) that was the primary reason Apple abandoned that hedge.

arrty889mo ago

I concur. I just upgraded from m1 air with 8gb to m4 with 24gb. Excited to run bigger models.

diggan9mo ago

> m4 with 24gb

Wow, that is probably analogous to 48GB on other systems then, if we were to ask an Apple VP?

1 more reply

dchest9mo ago

It's 4-bit quantized (Q4_K_M, 2.5 GB) and still works well for this task. It's amazing. I've been running various small models on this 8 GB Air since the first Llama and GPT-J, and they improved so much!

macOS virtual memory works well on swapping in and out stuff to SSD.

j / k navigate · click thread line to collapse