undefined | Better HN

0 pointsJedd1y ago0 comments

This feels like a weird complaint, given you started by saying it was 24GB, and then argued that the person who told you it was actually 32GB was making that up.

0 comments

behnamoh1y ago

Looks like Nvidia did a simple calculation:

    32GB (for 5090) / 24GB (for 4090) ≃ 1.33

Then multiply 4090's price by that:

    $1500 × 1.33 ≃ $2000

All else equal, this means that price per GB of VRAM stayed the same. But in reality, other things improved too (like the bandwidth) which I appreciate.

I just think that for home AI use, 32GB isn't that helpful. In my experience and especially for agents, models at 32B parameters just start to be useful. Below that, they're useful only for simple tasks.

JeddOP1y ago

Yes, home / hobbyist LLM users are not overly excited about this, but

a) they are never going to be happy,

b) it's actually a significant step up given the bulk are dual-card users anyway, so this bumps them from (at the high end of the consumer segment) 48GB to 64GB of VRAM, which _is_ pretty significant given the prevalence of larger models / quants in that space, and

c) vendors really don't care terribly much about the home / hobbyist LLM market, no matter how much people in that market wish otherwise.

j / k navigate · click thread line to collapse