Now it's probably because people don't like "traffic" outside their homes, but basically yes, that's why single-family zoning is still in place. It's extremely silly to want to ban eg corner stores in your neighborhood.
In SF where people have discovered "left-NIMBYism", people will now argue that keeping it is fighting racism, but then if you go into the suburbs they'll still happily argue the original position.