undefined | Better HN

0 pointsre-thc10d ago0 comments

Mistral Small 4 is already announced.

0 comments

MOE but 120B range. Man I wish it was an 80B. I have 2 GPUs with 62Gib of usable VRAM. A 4bit 80B gives me some context window, but 120B puts me into system RAM

Aerroon9d ago

Either some q3 or since it's a MoE, maybe a REAP version of q4 might work (or could be terrible, I'm not sure about REAP'd models).

j / k navigate · click thread line to collapse

0 pointsre-thc10d ago0 comments

Mistral Small 4 is already announced.

0 comments

androiddrew10d ago

MOE but 120B range. Man I wish it was an 80B. I have 2 GPUs with 62Gib of usable VRAM. A 4bit 80B gives me some context window, but 120B puts me into system RAM

Aerroon9d ago

Either some q3 or since it's a MoE, maybe a REAP version of q4 might work (or could be terrible, I'm not sure about REAP'd models).

j / k navigate · click thread line to collapse