You can want a diverse work force all you like but you aren't entitled to one. It's really quite simple, in America an employer doesn't get to pick the gender of his employees. Not to make it all one gender, not for diversity, not to have an exact 50-50 ratio, not to have all the typists be women and all the filers be men or vice versa.

In other words, you don't get to "add women to the team" because it's not your decision to make. If you're worried that your hiring process is biased against women, you're allowed to bend over backwards to make your job offers accessible to them, and go out of your way to make sure your workplace is not intolerable to whatever women work there already.