What happens today when some kids move road cones so they point to oncoming traffic as a prank? You can have a car react the same way I did in that situation - slow down, take in the entirety of the situation ("cones say X, but that leads you into oncoming traffic") and ask the operator for directions or maneuver accordingly.
Just because some Teslas are dumb enough to drive into barriers today doesn't mean we can't continue improving the technology.