It's a combination of two things, as far as I know.
1. The LLVM backend is slow.
2. The Rust frontend has historically abused the LLVM backend badly, by feeding it large amounts of barely optimized code. This is being worked on, but it's going to take a while.