I am a "street photographer" and have filled income gaps with wedding photography and film portraiture. Mysteriously, this issue has never come up with a woman.
Maybe if we were talking about the right to photograph a building from across the street, but if you think it's legal, ethical, or reasonable to take pictures of people they do not want you to take that are invasive of their privacy... Well... You will not win that case in a court of law, a debate on ethics, or in this forum of your peers.
Because it's a nonsensical talking point trying to turn everyone else into a victim but the actual victim.