I think that you would be hard pressed to find people who actually believe that. Most who point out problems with the CS environment probably feel no need to have any sort of robust view of innate differences between groups, and many are likely to be open to the possibility that some of the gender gap can be explained by such differences.
Note that much of the discussion of this issue has focused upon ways in which the CS culture is one which is overtly and unnecessarily hostile to women. It's not just that men vastly outnumber women in the field, it's that those men frequently engage in behaviors that are hostile, threatening, and demeaning to women. It is not implausible that these behaviors play a significant role in determining the gender imbalance in CS.