If I make something 1% faster on average, but now a random 0.000001% of its users see a ten-second stall every day, I lose.
It is tempting to think about it as a latency/throughput tradeoff. But it isn't that simple, the unbounded thrashing can be more like a crash in terms of impact to the system.
That's what you'd do if manually scheduling. Ideally the dynamic scheduler would do that on its own.
The other problem with spin-wait is that it overshoots, especially with an increasing backoff. Part of the overhead of sleeping is paid back by being woken up immediately.
When it's made to work, the backoff is often "overfit" in that very slight random differences in kernel scheduler behavior can cause huge apparent regressions.
