undefined | Better HN

0 pointsstuckinhell1y ago0 comments

100% reddit is full of people trying to solder more vram

0 comments

I've been wondering if you could just attach a chunk of vram over NVLink, since that's very roughly what FSDP is doing here anyways.

bick_nyers1y ago

The best NVLINK you can reasonably purchase is for the 3090, which is capped somewhere around 100 Gbit/s. This is too slow. The 3090 has about 1 TB/s memory bandwidth, and the 4090 is even faster, and the 5090 will be even faster.

PCIE 5.0 x16 is 500 Gbit/s if I'm not mistaken, so using RAM is more viable an alternative in this case.

Edit: 3090 has 1 TB/s, not terabits

j / k navigate · click thread line to collapse

0 comments

gaogao1y ago

I've been wondering if you could just attach a chunk of vram over NVLink, since that's very roughly what FSDP is doing here anyways.

bick_nyers1y ago

PCIE 5.0 x16 is 500 Gbit/s if I'm not mistaken, so using RAM is more viable an alternative in this case.

Edit: 3090 has 1 TB/s, not terabits

j / k navigate · click thread line to collapse