Under that argument - choosing not to mate is erasing people from the gene pool. Choosing who you mate with is also erasing people from the gene pool and denying people the possibility of mating with you at all is also erasing people from the gene pool. With the safe assumption that sperm and eggs have variations from one another - masturbating erases thousands if not millions of potential people from the gene pool and every period a female has without fertilization is another person missing from the gene pool.
Given the number of people on the planet - resulting in trillions (and trillions) of DNA combinations between mating pairs - millions of possible people are erased from the gene pool on the grounds of "not everyone can mate with everyone in a given lifetime, even if we wanted to". Not to mention all the potential genes that die each day.
I'm not sure I see your point.