While the evidence shown above is pretty clear that building a software package as a module provides the claimed benefits in terms of compile time (a reduction by around 10%, see Section 5.1.1) and perhaps better code structure (Section 5.1.4), the data shown in Section 5.1.2 also make clear that the effect on compile time of downstream projects is at best unclear.
One part of me agrees with (both from the paper)
> For example, putting a specific piece of code into the right place in each file (or adding necessary header files, as mentioned in Section 5.2) might take 20-30 seconds per file – but doing this for all 1051 files of deal.II then will take approximately a full day of (extremely boring) work. Similarly, individually annotating every class or function we want to export from a module is not feasible for a project of this size, even if from a conceptual perspective it would perhaps be the right thing to do.
and
> Given the size and scope of the library, it is clear that a whole-sale rewrite – or even just substantial modifications to each of its 652 header and 399 implementation files – is not feasible
but another part knows that spending a few days doing such ‘boring’ copy-paste work like that often has unexpected benefits; you get to know the code better and may discover better ways to organize the code.
Maybe, this project is too large for it, as checking that you didn’t mess up things by building the code and running the test suite simply takes too long, but even if it seems to be, isn’t that a good reason to try and get compile times down, so that working on the project becomes more enjoyable?
See VC++ devblogs and CppCon/C++Now talks from the team.
Pre-compiled headers have only worked well on Windows, and OS/2 back in the day.
For whatever reason UNIX compilers never had a great implementation of it.
With exception of clang header maps, which is anyway one of the first approaches to C++ modules.
Then around 1995 I got access to HP-UX and native compiler there and GCC. Nobody heard about precompiled headers and people thought the only way to speed up compilation was to get access to computer with more CPUs and rely on make -j.
And then there was no interest to implement precompiled headers from free and proprietary vendors.
The only innovation was unity builds when one includes multiple C++ sources into super-source. But then Google killed support for it in Chromium claiming that with their build farm unity builds made things slower and supporting them in Chromium build system was unbearable burden for Google.
We are commonly working with games that come with a custom engine and tooling. Compiling everything from scratch (around 1M lines of modern C++ code) takes about 30-40 seconds on my desktop. Rebuilding 1 source file + linking comes in typically under 2 seconds (w/o LTO). We might get this even lower by introducing unity builds, but there's no need for that right now.
Header units are basically chained PCHs. Sadly they are hard to build correctly at the moment.
1) modules only really help address time spent parsing stuff, not time spent doing codegen. Actually they can negatively impact codegen performance because they can make more definitions available for inlining/global opts, even in non-lto builds. For this reason it's likely best to compare using thin-lto in both cases.
2) when your dependencies aren't yet modularized you tend to get pretty big global module fragments, inflating both the size of your BMIs and the parsing time. Header units are supposed to partially address this but right now they are not supported in any build systems properly (except perhaps msbuild?). Also clang is pretty bad at pruning the global module fragment of unused data, which makes this worse again.
They are supported in build2 when used with GCC (via the module mapper mechanism it offers). In fact, I would be surprised if they were supported by msbuild, provided by "properly" we mean without having to manually specify dependencies involving header units and without imposing non-standard limitations (like inability to use macros exported by header units to conditionally import other header units).
It’s just more heavy clunky abstractions for the sake of abstractions.
- A component is a collection of related code.
- The component has an interface and an implementation.
- The interface is a header file (e.g. *.h) that is included (but at most once!) using a preprocessor directive in each dependent component.
- The header file contains only declarations, templates, and explicitly inline definitions.
- The implementation is one or more source files (e.g. *.cpp) that provide the definitions for what is declared in the header, and other unexposed implementation details.
- Component implementations are compiled separately (usually).
- The linker finds compiled definitions for everything a component depends upon, transitively, to produce the resulting program/dll.
So much can go wrong! If only there were a notion of components in the language itself. This way we could just write what we mean ("this is a component, here is what it exports, here are the definitions, here is what it imports"). Then compiler toolchains could implement it however they like, and hopefully optimize it.
It is no accident that Ada, Java, .NET, and oldies like Delphi, Eiffel, Modula-2 and Modula-3 have similar approaches.
Even the way D and Python modules and packages work, or the whole crates and modules approach in Rust.
Naturally folks not used to Web scale don't get these kind of features.
Perhaps we should use LLMs to convert all the legacy programs written in Fortran or COBOL into modern languages.
No, LLMs are not good at refactoring.