Given (at least the USA's) increasingly polarized population, I don't think it's at all true that people agree closely about what should be funded, and I'll admit that fact makes my argument weaker: The danger of a particular wealthy person "donating to evil" is similar to the danger that the majority of the country votes to "fund evil."
I also agree that wealthy folks spending their wealth on luxury yachts while the public suffers is also something to worry about. Who knew? Gargantuan wealth inequalities are mostly downside for everyone but the wealthy!