Most cryptography experts are probably not experts in quantum computers as well.
We already know the algorithm to break RSA with a quantum computer. We just don't have the hardware yet. Nobody knows when the hardware will be available but a lot of entities are working on it.
It's common in cryptography to mitigate attacks that are known but not feasible without further advances in hardware or algorithms. Nobody wants to wait until an attack is successful. That's why NIST is already working on post-quantum cryptography standardization:
https://csrc.nist.gov/projects/post-quantum-cryptography/pos...