Unfortunately I'm not able, in good faith, to take a stand for or against your argument. Consider this conceding the argument, but not being entirely convinced (allow me to explain).
Initially, I would like to reject it. Because as a libertarian-leaning trans, I fall under both examples you cited. Concern over potential abuse of PII, government overreach, and sexual identity (the concern over gender identity is similar in that regards). But I also fall under the "white, male" label.
However, I do understand the argument that a more targeted individual may be more capable of identifying potential issues. I feel this is contextual and often results in too many "maybes", "potentials", and "possibilities" to be entirely convincing.
"They maybe might have the potential to maybe see a possibility for something that might have the potential to maybe have the possibility of being abused." is not something I find convincing. Though it is technically correct and I have to concede that.