Rust-Written Linux Scheduler Showing Promising Results for Gaming Performance (opens in new tab)

I believe that some hype / over-enthusiasm is acceptable, but I'm a bit more worried when a given community uses it as some kind of organized propaganda, arriving to excesses like saying that writing code in non-safe languages should be considered immoral, and also never admitting that Rust is a compromise in the design space like anything else. So ok for the hype, but sometimes I see some excesses. Anyway this is just my very limited opinion, I don't claim to be right, I just believe my feeling represents some part of people here.

Especially when the hype is added by a third party. This is an article about a hobby project written in a few weeks and yet the headline suggests it is somehow a serious contender for being added to Linux (right now at least. Maybe something grows out of this, most likely not)

> I would just acknowledge the hype, nod in disagreement, and move on.

Which is exactly what he did, in addition to also acknowledging the pattern of the Rust community using hype to attract attention. He didn't force anything onto anyone, he just expressed his opinion.

Only if it doesn't harm your cause, which it does.

Writing oo style code in c is extremely common. At least where I work we often pass around structs of function pointers to accomplish roughly the same thing a vtable would give you and name methods such as foo_verb for methods that operate on struct foo.

An algebraic data type is just a c union with an auto generated flag and moving some type validation to compile time. Unions used in this manner are quite common in C. I do think the increased ergonomics and safety exist, but only when paired with other features like pattern matching. Selling algebraic data types alone as the major novel feature improvement is a bit misleading and dismissive of existing c features.

I do agree that additional compile safety in rust makes it far easier to confidently refactor without introducing bugs. Accomplishing the same in C requires a lot of unit tests which add maintenance overhead. Python is a more extreme example of that playing out. That all said, I don't think it's necessarily relevant to writing a performant scheduler.

Rust has many strengths and I endorse using it over C any day. That said, the way it's marketed feels misleading and gives experiences c developers bad vibes.

> Do you write OO on C?

Yes, have a look at GTK, GOBject and parts of Gnome.

> Do you write short optimized array-oriented code on C?

If you want to stay close to C you'd use something like SAC [1] but no, pure C is not an array programming language.

> People write C how people write C, the way C make them write it.

C is sometimes called 'structured assembly' for a reason: it is a toolbox which can be used to construct things the way you see fit. This does mean you need to involve yourself more with certain implementation details since C itself does not force you to use any specific paradigm and as such does not provide you with the basic tenets of those paradigms. If you want to do OO in C you'll have to provide a pointer to the object you're working on in any function call related to that object since C does not assume there to be a 'current object'.

Does this mean C is the most optimal language to do OO programming or array programming? No, clearly not, this is why languages like C++/Java and APL were created. On the other hand it does mean that it is possible to do these things in C and - given the success of Gnome and GTK - doing so can be a viable proposition. The advantage of using C is that it is nearly universally portable, more so than many other languages.

So yes, use of the freedom C provides (while being unsafe) you have the ability to implement whatever complex algorithm you can envision is actually true in that it is possible to do so. You may not want to use C for these purposes but that is irrelevant when considering whether it is possible to do so.

[1] https://www.sac-home.org/index

OO and other paradigms are not algorithms. They are particular ways of organizing code. Any algorithm can be implemented with OO, FP, or plain procedural style. Objects can be closures. Loops can be TCO recursion.

> I would not advise anyone to run a Linux scheduler I wrote and tested in C. However, if I wrote and tested one in Rust, I might not feel bad letting people use it.

For something as complex and sensitive as a kernel scheduler, i think "who" wrote the scheduler (as in how much experience writing scheduler), and what software dev. practices (especially how that thing was tested) and far better predictor than just the language used. I would actually go as far as saying that the language used might not even be a predictor at all.

No amount of rust safety would prevent things like dead/locks, quadratic algorithms in weird cases, unreleased resources etc... etc...

> All this news tells us is that a Rust implementation can compare to a C implementation in this field.

That's what the news "implied", but that's not actually what the news says. And i think that's what people are trying to call out.

The dev. implemented a prototype scheduler in rust, and in a very contrive case it does better than "a" C scheduler. The implementation are probably using different algorithm, they probably making different tradeoffs, we have no idea how safe and bug free his implementation his, and not even how safe it's (how much unsafe block is in that thing).

As an exercise to show that rust is a viable language to do kernel development in term of mature tool chain and good integration with kernel API sure. But as tool to compare C vs rust for kernel dev... this is pretty much worthless.

"Promising results for gaming performance" is not the kind of phrasing you use for toy projects though.

Rust programs are safer than C programs but almost universally more complicated to write compared to C with equivalent high level libraries for data structures and other operations. So I can't see how your argument applies here.

It's not just attention grabbing. So far, almost all rust development I've seen has been in drivers. Getting something as core as scheduler replaced is a very interesting event.

eBPF doesn't run in user space in the context of eBPF in the linux kernel. It's verified so that the kernel can be sure it won't loop forever and then gets JIT'ed and run in kernel space.

There are some user space BPF vms like https://github.com/iovisor/ubpf and Solana.

This is such a core idea that as you find it really surprising how much it gets overlooked.

IMO, another important aspect of rewrite is that it's usually pretty easy to get 70% of the functionality for 40 % of the work. But as one approaches 100% feature parity , plus handling all the corner cases, the transition from prototype to production ready things equalize pretty fast.

Not to mentioned the unknown unknown that the new language might also bring.

I think it's more because a game is very sensitive to jitter in CPU performance, and can reflect pretty well the responsiveness of the machine with a heavy load in the background. You can replace the game with scrolling a webpage, using slack during compile times, etc. With a game, you get a hard number (FPS) and a very visual indication of how well the load is doing (e.g. stuttering is very noticeable and is a pretty good indicator). So the game is not the point, it's IMHO a visualization of the effectiveness of the scheduler.

It's akin to running any CPU intensive task while playing a game... The hope would be to make the most progress possible without stalling the foreground process.

You have to do something while compiling a Linux kernel, might as well play a game as (I imagine) it can take a while :)

given the video literally showcases how to change the scheduler at runtime while the game is running, it's actually great. imagine Steam loading a new scheduler when a game starts? this way we could all the time run the best scheduler for the workload without issue.

The linux kernel has no concept of "active window process". It has the concept of "current process", and deciding which process that is, is exactly the job of the scheduler. :P

The scheduler doesn't need to know which process is associated with the currently active windows.

Simple heuristics can get you which to "reasonably" guesstimate which process are interactive and which aren't. For example, every X and descendant, or everything waiting on keyboard/controller or any other input.

I think the linux kernel historically did not want to prioritize interactive processes like windows and macos does.

The other thing is of course to reduce the CPU quanta, trading latency for throughput. I think most modern distribution do ship with better quanta for smooth GUI/desktop behavior.

One would think that at least a rewrite-from-scratch effort like Wayland would have come up with a solution.

Unfortunately the same problem also exists on other systems. Visual Studio even had to add a feature "Run build at low process priority" for the system to remain usable during builds:

https://devblogs.microsoft.com/cppblog/msbuild-low-priority-...

Hah, I should have spent 120 seconds searching :)

Correct -- the purpose of the watchdog is to account for buggy schedulers that fail to schedule runnable tasks. scx_rustland has special logic to track the user-space scheduling task, though. If it incorrectly failed to schedule it when it had no more runnable tasks to run, then the watchdog would eventually kick in, boot out scx_rustland, and revert back to EEVDF.

>There are absolutely things I'd be comfortable writing in Rust and not in C because I'd very likely mess up the C implementation. Rust encourages and lowers the cost for certain algorithms and implementations.

I share your sentiment, but that's not really relevant here. The kernel portion of the scheduler is written in C -- as are all of the other example (and production) schedulers we wrote. The BPF verifier ensures that the program is safe and can't crash the kernel, and we have a watchdog that will boot out a buggy scheduler that fails to schedule tasks in a timely manner.

>But even apart from that, the implementation being different doesn't matter much I think? I think it's more about there being a compelling component existing in rust, and less about whether it could be a different language or not.

An understandable point of confusion, but this is not the case. The fact that the user space portion of scx_rustland is written in rust is anecdotal. We have other schedulers which are entirely contained in the kernel, and others which have rich user space logic written in C. Many of them outperform EEVDF in a lot of different scenarios. By way of example, we're running a C scheduler now for Meta web workloads because it outperforms EEVDF by several percent for both latency and throughput.

> I think it's more about there being a compelling component existing in rust

Isn't the interesting part that there's a scheduler that's way better particularly in this use case? Linux is taking in Rust here and there already, I don't think that alone is interesting and the parent point is that the same thing could have been written in C.

There are compelling reasons to choose Rust, but it's not magically fixing performance by virtue of not being C. In which case, the implementation is the story.