There's really an amazing lack of imagination here, both from a threat avoidance perspective and a potential awesomeness one.
The deployment model is this: one large webmail provider starts doing PGP by default via its webclient. Maybe it provides your with private keys, maybe it doesn't. Fact is that it doesn't much matter, because as soon as a large webmail provider starts doing PGP/PKI, the two biggest problems with adoption (namely, that there's no one to use it with, and it's kind of a pain to use anyhow) are basically solved. And as soon as this happens, there starts being a competitive market where providers can begin improving on each other's implementations. Any provider that doesn't give users their private keys won't have much of an ethical argument for doing so, and so it probably would, anyway. There will, as always happens, be a feature war, except with PGP involved some of that war will involve privacy/encryption/reliability concerns.
(PGP also makes spear phishing much harder).