You're spot on. It's possible to trace every wallet a coin passed through. If your coin ever passed a blacklisted address, it's tainted and nobody will accept it. Exchanges are already rejecting deposits containing coins that passed through mixing services.

Monero is fungible because it's private. Every transaction is signed by lots of users and nobody can tell where the coins are actually coming from. There's no way to tell them apart.