Really? I seem to recall lld being comparable to cp in speed, with the caveat that compacting debug strings can take a long time (if you enable that option).
For example, on macOS in debug builds, the compiler and linker are reasonably fast, but then 2/3rds of the compilation time is spent in "dsymutil", presumably chewing through megabytes of the debug info.