Why are you so certain you couldn't you afford them? You were able to purchase a 200K house many years ago. Now you just need another 200K to buy that bigger house — and it's a bigger house! You did it once long ago, you can probably do it again even easier (in addition to everything else, you have a huge downpayment now). And no, you typically don't pay capital gains taxes against your primary residence.
In any case, the point is that you're in a way better place than if you hadn't made that 200K purchase. That's what I mean by "floating on the water" — you have a stake in the market so now as it moves, so does your asset.
