undefined | Better HN

0 pointsGracana1y ago0 comments

It's "easy" if you have a place to build an open frame rig with riser cables and whatnot. I can't do that, so I'm going the single slot waterblock route, which unfortunately rules out 3090s due to the memory on the back side of the PCB. It's very frustrating.

0 comments

diggan1y ago

I think parents point is that NVLink no longer ships with consumer cards. Before you could buy two cards + a cable between them, and software can treat them as one card. Today you need software support for splitting between the cards, unless you go for "professional" cards or whatever they call them.

GracanaOP1y ago

Maybe that's what they meant, and it'd be cool if nvidia still offered that on consumer cards, but thankfully you don't need it for LLM inference. The traffic between cards is very small.

diggan1y ago

Isn't the issue that the software needs to explicitly add support for it now, compared to yester-yesterday when you could just treat them as one in software?

numpad01y ago

There was a rumor that 5090 or 5090D for China may or may not come with multi-GPU software locked. I think GP's referring to that. It's not clear if it is the case with retail cards.

j / k navigate · click thread line to collapse

0 comments

diggan1y ago

GracanaOP1y ago

Maybe that's what they meant, and it'd be cool if nvidia still offered that on consumer cards, but thankfully you don't need it for LLM inference. The traffic between cards is very small.

diggan1y ago

Isn't the issue that the software needs to explicitly add support for it now, compared to yester-yesterday when you could just treat them as one in software?

numpad01y ago

There was a rumor that 5090 or 5090D for China may or may not come with multi-GPU software locked. I think GP's referring to that. It's not clear if it is the case with retail cards.

j / k navigate · click thread line to collapse