undefined | Better HN

0 pointsrisho1y ago0 comments

how much vram do you need for 4-bit llama 405?

0 comments

405 billion * 4 bits = approximately 200 GB. Plus extra for the amount of context you want.

j / k navigate · click thread line to collapse

0 pointsrisho1y ago0 comments

how much vram do you need for 4-bit llama 405?

405 billion * 4 bits = approximately 200 GB. Plus extra for the amount of context you want.

j / k navigate · click thread line to collapse