But Mistral has fall really far behind since 2025Q3. It seems they can't get good reasoning models working at even medium context sizes, which is necessary to be at the table right now.
Gemma4 and Qwen3.6 are currently best in the small size; Mistral's "small" model has ~4x the parameter count at 120B and isn't even competing with models a quarter its size.
Back one year ago with Mistral Small 3.1 they were keeping up, but they've fallen into irrelevancy right now.
If Mistral seriously wants to play the on-prem and small task-specific model game, a decent proxy would be to build models that get the r/localLlama crowd excited
I am wondering what is keeping them back, though: Money? Compute? Skills? Training data? My fear is that you are really only getting really good models by training on very dubious data (outputs from the frontier models etc) and that Mistral is too European and too enterprisey to take those risks.
Or at least there’s been a lot of noise about that.
This is tangential: and forgive my ignorance here, but is there an inherent reason why there aren't smaller, focused models from the frontier model providers?
I'm thinking something like a software-specific subset of Opus that is the default for use in Claude Code. Smaller, cheaper to deploy and consume, maybe faster.
Even though Mistral 4 has 6B active parameters per token (allowing 3-3.5 per token parameters to be loaded on a 4090), the ~240GB download + storage is pushing the limits of being able to try this out locally, especially if you are downloading and evaluating multiple models.
It also makes it harder for other people to make downstream finetunes like with what happened with the older Mistral/Magistral models.
Fully agree to your point though, Mistral in general is far behind where I'd expect and Qwen in particular is crushing it at the smaller sizes.
Personally, I'd consider anything 20B params and above a "medium" model. Small being <20B and large >100B. I think obviously we can get to the huge 1-2T param models, but frankly the margin of accuracy improvement for the speed hit is kinda insane (1-2% for many metrics).
1. tiny <2-3B -- easily runnable on lower-spec hardware
2. small 4-8B -- runnable on 8GB GPUs
3. medium 9-12B -- runnable on 12GB GPUs
4. large 13-24B -- runnable on 16GB (for the lower end models) and 24GB GPUs
5. very large 25-32GB -- runnable on 32GB GPUs
6. huge >32GB -- not easily runnable on consumer GPUs without compromising performance (offloading layers to the CPU/RAM), quality (heavy quantization, esp. at <= Q4), or price (investing in multi-GPU setups and/or server-grade hardware).
You could possibly split huge down further, as 70GB models (e.g. llama 3) are easier to get working than >120GB models and 1TB models are completely intractable.
1. tiny <2-3B -- could run in a browser even, mac neo
2. small 4-8B -- last of browser options, MacBook Air base
3. medium 9-24B -- 32GB machine, air or pro notebook or mini
4. large 25-48B -- 64GB, pro notebook or mini
5. x-large 49-100B -- 128GB MacBook Pro or Studio
6. Huge > 100B -- 256/512GB Mac Studio
Foundation model labs should be building very large reasoning models, then leaving it to the community to distill them down.
You can't scale a small model up, but you can scale a small model down.
I'm convinced the only way we'll have a seat at the table in the future and avoid total runaway takeoff is if there are very large models within 80% of the capabilities of the frontier models. Tiny RTX models do diddly squat to remain competitive.
Build open weights models for running on H200s. I'll spin them up on RunPod or Lambda.
I have used Mistral models out of pure ideology for web agents and the like which aren't doing a lot of heavy lifting.
Our evals are pretty complex so we only recently started testing ~30B class models, which are now becoming quite smart (on par with the frontier from 1 year ago). Mistral is far behind, but I'm rooting for them.
Data at https://gertlabs.com/rankings
