I think that is a big dump to make. Just because someone is libertarian and recognizes that women and welfare recipients are the biggest challenge, does it mean that their proposed solution is to take their vote away.
I'm willing to agree that by the nature of his power and position, he is insulated from the negative externalities of those he politically funds. The jump that I make, is that he ultimately wants the government out of those areas too.