undefined | Better HN

0 pointsbrookst1y ago0 comments

And, as I've noted a couple of times in this thread, how many times have we heard that Moore's law is dead and compute has hit a wall?

0 comments

moffkalast1y ago

Well according to Nvidia you can just ignore Moore's law and start requiring people to install multi kilowatt outlets just for their cards. Who needs efficiency amirite?

jmward011y ago

I'm not an apple fan (as I type on a mac that I am forced to use) but I gotta applaud their push for power efficiency. NVIDIA actually -does- have a few cards they make that really improve power efficiency but then they generally hamstring them with a lack of memory. NVIDIA is really good at making their high-end cards the only viable choice but I think that will backfire on them as people like me, that value quiet, cool and efficient over 25% faster inference start taking any viable alternative that comes out.

llm_trw1y ago

Memory is king.

Anything that has more memory and adequate compute will win the coming AI wars.

At the rate at which power consumption is growing now that the shortage of current gen cards has started to work itself out people are realizing they need a fleet of nuclear reactors to keep the data centers running. This is not something that's getting fix with the coming generation, if anything it's worse.

j / k navigate · click thread line to collapse