Yes, it's a shitty problem, but it's because the industry is shit. There must have been extensive studies on ad UX but I see little improvements because so many companies just wanted to sell ads to make $$. Take Forbes and similar new sites - freakin' annoying. But NYT ads are good, at least not as distracting as other news sites seen to be.
Since no one wants to host the ad platform on their own, how do you think you will get your idea across? I think what you really meant is build a better ad platform, not self-hosting ad platform.