What does that mean for a linker? If you ship a binary linked with an AGPL linker you need to offer the source of the linker? Or of the program being linked?
AGPL is avoided like the plague by big corps: same big corps are known for having money to pay for licenses and sometimes (yes, I look at you Amazon) being good at deriving value from FLOSS without giving back.
iirc AGPL was used so everyone can just use it, big biz is still compelled to buy a license. this has been done before and can be seen as one of the strategies to make money off FLOSS.
This is not theoretical; it happens quite frequently. For toolchains, in particular I'm aware of how Apple (not that they're unique in this) has "blah blah open source" downloads, but often they do not actually correspond with the binaries. And not just "not fully reproducible but close" but "entirely new and incompatible features".
The ARM64 saga is a notable example, which went on for at least six months (at least Sept 2013 to March 2014). XCode 5 shipped with a closed-source compiler only for all that time.
So the benefit will be barely noticable. As another comment points out, LTO should only be used when you need a binary optimized to within an inch of its life, such as a release copy, or a copy for performance testing.
Specifically, if LTO is so important that you need to be using it during development, you likely have a very exceptional case, or you have some big architectural issues that are causing much larger performance regressions then they should be.
And for realease builds link time is hardly a concern.
That doesn't extinguish the prior versions under the prior license, but it does allow a project to change its license.
For those on macOS, Apple released a new linker about a year or two ago (which is why the mold author stopped working on their macOS version), and if you're using it with Rust, put this in your config.toml:
[target.aarch64-apple-darwin]
rustflags = [
"-C",
"link-arg=-fuse-ld=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld",
"-C",
"link-arg=-ld_new",
]I have the command line tools installed and I only have /usr/bin/ld and /usr/bin/ld-classic
External libs are most often linked dynamically these days, so they don't need to be built from source, so eliminating the linker doesn't pose a problem for non-open source dependencies. And if that's not enough letting the compiler also consume object files could provide for legacy use cases or edge cases where you must statically link to a binary.
> Developers sometimes experience trouble debugging the quarter-million line amalgamation source file because some debuggers are only able to handle source code line numbers less than 32,768 [...] To circumvent this limitation, the amalgamation is also available in a split form, consisting of files "sqlite3-1.c", "sqlite3-2.c", and so forth, where each file is less than 32,768 lines in length
Apologies for the typo. And now it is too late to edit the post.
Even modest statically linked rust binaries can take a couple of minutes in the link stage of compilation in release mode (using mold). It’s not a rust-specific issue but an amalgam of (usually) strictly static linking, advanced link-time optimizations enabled by llvm like LTO and bolt, and a general dissatisfaction with compile times in the rust community. Rust’s (clinically) strong relationship with(read: dependency on) LLVM makes it the most popular language where LLVM link-time magic has been most heavily universally adopted; you could face these issues with C++ but it wouldn’t be chalked up to the language rather than your toolchain.
I’ve been eyeing wild for some time as I’m excited by the promise of an optimizing incremental linker, but to be frank, see zero incentive to even fiddle with it until it can actually, you know, link incrementally.
Additionally, the acceptance of binary libraries across the C and C++ ecosystem, means that more often than not, you only need to care about compiling you own application, and not the world, every time you clone a repo, or switch development branch.
2015(?): Lld a drop in replacement linker, at least 2x as fast as Gold
2021: mold, a new linker, several times faster than lld
2025: wild, a new linker...
The book Linkers and Loaders by John Levine.
Last book in the list here:
https://www.johnlevine.com/books.phtml
I had read it some years ago, and found it quite interesting.
It's a standard one in the field.
He has also written some other popular computer books (see link above - pun not intended, but noticed).
To use it with Rust, this can probbaly also work using gcc as linker driver.
In project's .cargo/config.toml:
[target.x86_64-unknown-linux-gnu]
rustflags = ["-C", "link-arg=-fuse-ld=wild"]
As for why Rust uses gcc or clang to invoke the linker rather than invoking the linker directly - it's because the C compiler knows what linker flags are needed on the current platform in order to link against libc and the C runtime. Things like `Scrt1.o`, `crti.o`, `crtbeginS.o`, `crtendS.o` and `crtn.o`.
Instead of renaming and passing -B in, you can also modify the GCC «spec» file's «%linker» section to make it point to a linker of your choice, i.e.
%linker:
/scratch/bin/wild %{wild_options}
It is possible to either modify the default «spec» file («gcc -dumpspecs») or pass your own along via «-specs=my-specs-file». I have found custom «spec» files to be very useful in the past.
The «spec» file format is documented at https://gcc.gnu.org/onlinedocs/gcc/Spec-Files.html
May be it's worth filing a feature request for gcc to have parity with clang for arbitrary linkers?
"Because there's low hanging concurrent fruit that Rust can help us get?" would be interesting but that's not explicitly stated or even implied.
But in general, I'd guess just different design decisions. As for how this might be related to Rust - I'm certain that were Wild ported from Rust to C or C++, that it would perform very similarly. However, code patterns that are fine in Rust due to the borrow checker, would be footguns in languages like C or C++, so maintaining that code could be tricky. Certainly when I've coded in C++ in the past, I've found myself coding more defensively, even at a small performance cost, whereas with Rust, I'm able to be a lot bolder because I know the compiler has got my back.
Perhaps it is worth repeating the experiment with heavy MLoC codebases. jmalloc or mimalloc.
... however...
> code patterns that are fine in Rust due to the borrow checker, would be footguns in languages like C or C++,
That "dig" is probably not true. Or rather, your very conflation of C and C++ suggests that you are talking about the kind of code which would not be used in modern C++ of the past decade-or-more. While one _can_ write footguns in C++ easily, one can also very easily choose not to do so - especially when writing a new project.
https://news.ycombinator.com/item?id=33330499
NB. This is not to suggest wild is bloated. The issue if any is the software being developed with it and the computers of those who might use such software.
"... I have 16 GB of ram, I can't upgrade it..."
(Personally, upgrading my laptop to 64GB at the expense of literally everything else was almost a great decision. Almost, because I really should have splurged on RAM and display instead of going all-in on RAM. The only downside is that cleaning up open tabs once a week became a chore, taking up the whole evening.)
On modern large hardware architecture, for executable files/dynamic libraries, ELF(PE[+]) has overkill complexity.
I am personnally using a executable file format of my own I do wrap into an "ELF capsule" on linux kernel. With position independent code, you kind of only need memory mapped segments (which dynamic libraries are in this very format). I have two very simple partial linkers I wrote in plain and simple C, one for risc-v assembly, one for x86_64 assembly, which allow me to link into such executable file some simple ELF object files (from binutils GAS).
There is no more centralized "ELF loader".
Of course, there are tradeoffs, 1 billion times worth it in regards of the accute simplicity of the format.
(I even have a little vm which allows me to interpret simple risc-v binaries on x86_64).
Compilers take the code the programmer writes, and turns it into things called object files. Object files are close to executable by the target processor, but not completely. There are little places where the code needs to be rewritten to handle access to subroutines, access operating system functionality, and other things.
A linker combines all these object files, does the necessary rewriting, and generates something that the operating system can use.
It's the final step in building an executable.
--
More complicatedly: a linker is a little Turing machine that runs over the object files. Some can do complicated things like rewriting code, or optimizing across function calls. But, fundamentally, they plop all the object files together and follow little scripts (or rewrites) that clean up the places the compiler couldn't properly insert instructions because the compiler doesn't know the final layout of the program.
Symbols would be resolved based on an index where only updated object files are reindexed. It could also eagerly relocate in the background, in order depending on previous usage data.
This would basically make a copyless lazy incremental linker.
By the time you have addressed these, you'll find yourself building a microkernel system with a collection of independent servers and well-defined interaction protocols. Which isn't necessarily a terrible way to assemble something, but it's not quite where you're trying to go...
Not exactly this, but Google's Propeller fixes up ("relinks") Basic Blocks (hot code as traced from PGO) in native code at runtime (like an optimizing JIT compiler would): https://research.google/pubs/propeller-a-profile-guided-reli...
I thought a lot of its proofs were done at compile time not link time.
Can someone explain what is so special about Rust for this?
https://doc.rust-lang.org/book/ch16-00-concurrency.html
So the logic would go:
1. mold doesn't do incremental linking because it is too complex to do it while still being fast (concurrent).
2. Rust makes it possible to write very complex fast (concurrent) programs.
3. A new linker written in Rust can do incremental linking while still being fast (concurrent).
EDIT: I meant this originally, but comments were posted before I added it so I want to be clear that this part is new: (Any of those three could be false; I take no strong position on that. But I believe that this is the motivating logic.)
1. Rust's well designed type system and borrow checker makes writing code that works just easier. It has the "if it compiles it works" property (not unique to Rust; people say this about e.g. Haskell too).
2. Rust's type system - especially its trait system can be used to enforce safety constraints statically. The obvious one is the Send and Sync traits for thread safety, but there are others, e.g. the Fuchsia network code statically guarantees deadlocks are impossible.
Mold is written in C++ which is extremely error prone in comparison.
Linking is often a very notable bottleneck for debug binaries and mold can make a big difference.
So interest in speeding up linking for Rust is expected.
By the way I had to lookup what incremental linking is, in practice I think it means that code from libraries and modules that have not changed won’t need to be re-packed each time which ch will save time for frequent development builds, it’s actually ingenious