I would love to not talk while driving. I would also like to not listen while driving. In fact, I would like to not drive while driving! That being said, I'm much less concerned about spending a minute talking to a dispatcher than the guy/gal flying down I-95 while breaking up with someone over the phone. (Speaking of which, at least with a phone you can hang up... but some passengers, man...)
Edit: I would also posit that in some cases, a short phone call can actually do a great deal to remove distraction if it is a settleable matter. The conversation you have in your head while driving may be just as bad as the one you'd have on the phone, except that it may dwell longer.