undefined | Better HN

0 pointsmake35mo ago0 comments

the models are huge, so not a single (latest gen) one can fit on a single GPU.

It's likely that these are small unpopular (non flagship) models, or that they only pack eg one layer of each model.

0 comments

Per the very short article, the solution was to pack multiple models per GPU.

make3OP5mo ago

yes but that could mean a layer per model

j / k navigate · click thread line to collapse

0 pointsmake35mo ago0 comments

the models are huge, so not a single (latest gen) one can fit on a single GPU.

It's likely that these are small unpopular (non flagship) models, or that they only pack eg one layer of each model.

Per the very short article, the solution was to pack multiple models per GPU.

make3OP5mo ago

yes but that could mean a layer per model

j / k navigate · click thread line to collapse