Shipping a compiler every six weeks (opens in new tab)

Over at the TypeScript compiler we recently slowed our release cycle from 2 months to 3 months specifically because of an observation noted here: nobody used our beta (or RC) builds. On a 2 month schedule, we had 1-2 prelease builds during any given 6ish week period. The faster releases we had before were great for us - if something wasn't ready, we could just sit on it till the next release, since they were so often. However because they were so often, we struggled with collecting feedback on prereleases - we always, consistently, got most regressions reported only after the full release. We didn't really like this so a few things were tried. First we added an earlier prerelease cut and feature freeze (the beta) - this made the release smaller (as the earlier freezes meant more time focused on regressions or on the next release), and we still didn't get any feedback. Some of our users gave us the feedback that they'd test against our betas, but only if we had less of them... So we lengthened our release cycle to try that. We've not done many on the longer 3 month cycles yet, so I can't say if it's helped or not yet, but from the feature development side, I can definitely say that the longer releases definitely slow down how quickly things are built on top of one another.

In parallel, we've also been improving our test infrastructure in ways similar to Crater. In the past few cycles we've added a bunch of tests which only exist to test compatibility - all of DefinitelyTyped is now tested on our builds, and a number of community projects that we've produced build scripts for are built as well. I would _love_ to be able to crawl GitHub and just build arbitrary stuff, but TS/JS build chains are almost never a simple `npm run build`, so the best we can get is, approximately, loading a repo into an editor and checking for a crash (which we do have a tool doing). The rust ecosystem's dogma around using `cargo build` and `cargo test` to handle _everything_ really does help make what they're doing possible.

The author is a bit inaccurate, depending on how you count things. At Microsoft we ship the C#, F#, and CB compilers on a cadence faster than 6 weeks; probably every 2 weeks on average. New language features don’t make it in to each of these releases, but bug fixes and performance improvements certainly do. These releases are kore driven by tooling evolution (VS updates, .NET SDK updates) than language evolution though.

At Google, we strive to ship a new Clang/LLVM toolchain to our C++ developers every week.

Two of our toolchain engineers gave a talk on this at CppCon. Check it out if you're interested: https://www.youtube.com/watch?v=zh91if43QLM

It's pretty cool to see a demonstrable example of how a solid testing framework can significantly boost development times. Especially one that includes end-to-end tests as well.

I've been involved in a lot of projects that rely entirely on unit tests and skimp on integration tests. Invariably the devs make up for this by doing a lot of manual testing, even though it is very time and labor intensive. People significantly underestimate the safety and velocity benefits of a solid suite of integration tests

This is the most inspiring thing I’ve read this week. I wish more would do this kind of thing.

The editions were a bad idea. And worse they led to completely shutting down the idea of a Rust 2.0 which dumps all the flawed ideas for which better solutions have been found. So they have to work twice as hard to continue maintaining code to support features that people shouldn't be using anymore anyway. If dropping support for unmaintained crates lead to an ecosystem where one could reasonably integrate crates one wants to use together that'd be way better than the situation we currently have where old crates still compile but nothing agrees on the proper way to do anything and a large number of crates can't update to the new idioms because they need to remain minimally usable in relation to slow moving or abandoned crates.

I am the maintainer of a Haskell like JIT compiler used for running complicated paring/rostering optimizations for major airlines. It has never had a bug in production thanks to 9000+ tests. We routinely add new features and push into production with no fear because of auto tests. This can be done in less than a day if urgent.