Runtimes/vms implement memory management, varius threading techniques and things like we see here
Maybe in the future we will entirely skip OS's overhead and run apps directly on HW and they will manager themselves more efficiently (their runtimes/vms like jvm clr)
That's one of the reasons unikernels seem to be a promising way forward. It opens up a bunch of opportunities, including language-based safety, opportunities for compile-time optimizations, and just seems to mirror more closely how we wish to run & deploy modern applications (declarative, immutable and ideally with a bare minimum of dependencies).
Beyond "kernel programming is hard", there are a few other reasons why it made sense for us:
- observability & maintenance: much easier to implement and ship this type of changes in userspace than rolling out a kernel fork. We also built custom AB infra to be able to evaluate these optimizations.
- the kernel is really good at making reasonable decisions at high-frequency based on a limited amount of data and heuristics. But these decisions are far from optimal in all scenarios. In contrast in user-space we can make better decisions based on more data (or ML predictions), but do so less frequently.
https://www.destroyallsoftware.com/talks/the-birth-and-death...
Predictive CPU isolation of containers at Netflix using a MIP solver - https://news.ycombinator.com/item?id=21116565 - Sept 2019 (21 comments)
Predictive CPU Isolation of Containers at Netflix - https://news.ycombinator.com/item?id=20096699 - June 2019 (1 comment)
You save money still, but you don’t solve your capacity problems by doing so.
