The key point here is leveraging, given that an average US mortgage is 6.5x to 20x.

And that leverage is broadly available to most people, supported (and funded) by government policies.

On the other hand, if you need housing (and who doesn't!), it's not a bad deal to pay for a house and "only" end up realizing (at a later sale) a couple percent of gains. Frankly I consider that a great deal! I wish I could buy a car and sell it 10 years later for higher than the purchase price. Or furniture. Or computers. Anything non-consumable, really.

Also probably important to consider that most people who buy homes cannot do so without a mortgage. So you're not even really comparing it to whatever else you might do with that money (well, aside from the down payment); it's just a pure expense. Sure, unless you have an interest-only loan, you're also putting principal into it every month (which you could put in the S&P instead), but that's sometimes better than giving a similar amount to a landlord, depending on housing economics in your area.

We shouldn't even have to think in terms of "breaking even" because housing should't be an investing tool. It's a shelter and a home.

Unfortunately, making "good investments" is necessary to stay above water in this precarious society, and our society is so fucking heartless that houses are one of the best investments a person can make. It's both bleak and absurd.

Typically there's much less tax involved when investing in real estate. Here (Australia) there's both no tax on the imputed rent or price appreciation if it's your own home.