Well of course they worry about it. I can ditch Google and go to Yahoo, DDG, Bing, hell Lycos might still be around, if the Google homepage takes too long to load. Obviously same for Yahoo. Neither of them produce or host the content (search, news) that you're going there for.
That's not quite the same situation as Twitter, GMail, Facebook and whatnot. Where else are you going to go? Your mail is in GMail, or your friends tweet is on Twiter, their wall is on Facebook and they are only there. The "instant load or I'm out of here" doesn't apply so much to them. It really only affects new users who have no other attachment to the service.
I doubt it's much of a problem for Twitter.