You can look up people even trying to detail their cars to make them cleaner and end up leaving “love marks.” It doesn’t matter how soft the thing you’re using is. It’s because the car has contaminants on it and by rubbing anything on the car, those contaminants end up scratching everything. It’s like when you’re at the beach and you’re trying to remove sand off your skin. You’re probably not aggressively rubbing it off or using much pressure but it still hurts. It’s the same with cars, it’s just that the rocks aren’t as visible to you. They will leave swirls and scratches though… which become noticeable.
I’ve had people just lean against my car when it wasn’t completely clean and completely ruin the paint requiring an entire 5 stage detail.