undefined | Better HN

0 pointsKaiserPro3mo ago0 comments

Depending on what you're doing its taking up to 8GPUs working in parallel to serve those queries.

0 comments

Yes but then the batch size is in 100s or even 1000s. These GPU doesn't serve just 1 user at a time.

j / k navigate · click thread line to collapse

0 pointsKaiserPro3mo ago0 comments

Depending on what you're doing its taking up to 8GPUs working in parallel to serve those queries.

Yes but then the batch size is in 100s or even 1000s. These GPU doesn't serve just 1 user at a time.

j / k navigate · click thread line to collapse