undefined | Better HN

0 pointsBaculumMeumEst1y ago0 comments

What is the best of the llama 3.1 models that I can fine-tune with a macbook m3 max w/ 96GB of ram?

0 comments

None unless you are prepared to spend 5+ years. Macs just don't have flops.

Really? You can't even fine-tune a quantized 8B model on such a machine? That's a bummer.

You can't really fine-tune quantized models as-is. Gradients require floating-point calculations at descent precision.

j / k navigate · click thread line to collapse

0 pointsBaculumMeumEst1y ago0 comments

What is the best of the llama 3.1 models that I can fine-tune with a macbook m3 max w/ 96GB of ram?

None unless you are prepared to spend 5+ years. Macs just don't have flops.

Really? You can't even fine-tune a quantized 8B model on such a machine? That's a bummer.

You can't really fine-tune quantized models as-is. Gradients require floating-point calculations at descent precision.

j / k navigate · click thread line to collapse