Even if he knew, why would anyone expect Elon not to lie about anything?

> The have plenty if data.

I don't think data is the problem either, but compute is: if you want to train your 5T params model like modern small models are being trained (with a thousands time more training tokens than params), that's an enormous training run.

Some interesting notes:

- Training a small model with large model output resulted in LESS improvement than distilling a less smart model onto the same small architecture [0]. We are starting to hit intelligence density limits in small models (<30B models may be nearing saturation now)

- good RL environments incidentally also make for good benchmarking

[0] https://arxiv.org/html/2502.12143v1

So where does humanity cap out? The statement more or less implies that there's a ceiling of our ability to train models which might be below what LLMs are capable of (e.g. not AGI but how good coding agents they might ever become, for example).

Xai paying cursor to train models with their data, tell us that having an agent tool like claude code is important for quality data acquisition. That’s why they recently shipped grok build

I think we will see insane SOTA models from xai in the next few months.