Yeah that makes sense to me. If you trust ssl. I usually assume that if some three-letter agency wants to hack my computers they are going to find a way and recent history has shown that SSL can be vulnerable too.
I think it's true that without certificate pinning (which you sound like you know about) the various government agencies may easily have people inside your certificate stores that can issue bogus certs. That we've never read of one of these attacks succeeding is further evidence that the conspiracy is working ;)