The argument in favor of Google Fonts and similar services is that visitors likely already have that font cached from visiting another site that uses the same font. It can also decrease bandwidth costs if that's a bottleneck.

That said, serving from your own domain is a great way to make sure that a third party can't break your fonts, in addition to what you mentioned about privacy.