Only like three towns are "heavily mapped" to the level required for a Google self-driving car to work. And the problem is that if the car relies on the map, your first car to see a road or a change in the road, won't know what to do.
"As far as you're concerned" doesn't mean a whole lot.