The closest to an alternative model that works here is wikipedia. They achieve it by having a different default for user generated content - that it doesn't remain unless it is strictly on topic (you can't just write your random thoughts and shitposts in an article) - and by having a social mission that attracts donations. (They also run incredibly lean, which other mass market sites could definitely learn from.)

Another alternative model being pursued is the federated approach. I like this approach in principle but I don't think it will ever reach the mass market hundreds of millions of active users position of the mainstream social media products. Its solution to the problem is that some subset of power users will self-host and absorb the costs, either via individual altruism or something, or by developing some other model. I think this will end up centralizing, with a small number of nodes hosting most of the usage, and probably eventually falling back to advertising to cover costs. But I dunno, we'll see, maybe this will work.

Another more successful model is to just not be mass market at all, like HN and other various message boards. This reduces costs both for hosting and moderation and can then be justified by some non-financial benefit (like tacit advertisement for YC in the case of HN, or by tight community camaraderie for niche message boards).

Then another model that works is subscriptions. This drops the active user count massively and generally makes it harder to get traction, but I think it works the best of any of these when the stars align. I'd rather be Netflix or the NYT than Twitter or Reddit.