Nothing is stopping you from building a castle on your property now. My guess is the difference in price is about 5-fold - and the castle will need tons of costly repairs to adapt to whatever the favored amenities are.

I've lived in really old buildings. Even if the bricks on the outside were over 100 years old it was extremely expensive to update the air conditioning and windows to be a good living experience. Why take such a superficial view of what is "built to last" just because the outside stays the same? Best case scenario for a long last building is a ship of theseus situation. And while it may be cost efficient for historic college campuses to renovate their old buildings continuously, chances are this not applicable to random middle class homeowners.