The main issue is that we can make more food, but we can't make more land.
We can make more buildings though. You'll notice that most of the appreciation in house prices lately has to do with land, not the buildings.
The fundamental error in our understanding of economics is treating non-produced goods like land as if they were "capital." There's nothing wrong with investing in producible goods -- in fact that's ideal. But when people gatekeep nonproduced goods, especially nonproduced goods that everybody needs to live (ie, land) - that's where everything breaks down.