If I'll want to refer to you and this conversation when talking to someone else, I don't need to refer to your gender at all. Just like I don't need to refer to your race, your social class or color of your hair. Stopping to consider whether I should use "he" or "she" (or maybe something else) is the exact opposite of efficiency in speech.
It seems to me that it's actually you who misunderstands the usefulness of specificity. It's not useful to be overspecific.
