Unfortunately, yes. Dropping a magnet onto a car and pulling it off, especially if not recently cleaned, will damage the paint to some degree. Maybe not enough for an average person to notice, but you really shouldn’t do this to other people’s cars.

Some people will get snide about anyone who cares about their car’s paint, but as someone who once bought a car I had to save a long time for and spent a lot of time with car care products I would be very sad if I saw you drop a magnet on to it and then pull it off without a second thought. Please don’t.

It won't really matter all that much, but it will have done more than 0 damage to the paintwork (since metal is hard and paint is soft). Worth noting that drivers are touchy and emotional, and can't be trusted not to murder you over perceived slights, so it's safest to stick to doing nothing. Stuff something under the windscreen wipers if you really must, and even that is risky.

A flexible fridge magnet is probably fine.

Seems like everyone here is assuming you used a 40lb neodymium magnet you dropped in the dirt first.

I like to assume the best in people.

Unless the cars are perfectly washed and clayed, even running a clean finger over a car is likely to introduce scratches. I just wouldn’t ever touch someone’s car.

You can look up people even trying to detail their cars to make them cleaner and end up leaving “love marks.” It doesn’t matter how soft the thing you’re using is. It’s because the car has contaminants on it and by rubbing anything on the car, those contaminants end up scratching everything. It’s like when you’re at the beach and you’re trying to remove sand off your skin. You’re probably not aggressively rubbing it off or using much pressure but it still hurts. It’s the same with cars, it’s just that the rocks aren’t as visible to you. They will leave swirls and scratches though… which become noticeable.

I’ve had people just lean against my car when it wasn’t completely clean and completely ruin the paint requiring an entire 5 stage detail.

As long as the car is dirty, then contact with it can damage the top coat. This is a lot more true if you need to drag or scrape the magnet to remove it.