The question that ultimately convinced me to just pay off my mortgage was "If someone gave you a house, free and clear, would you mortgage it to buy investments?". My gut reaction was "no, I would really like to have a free and clear house."
I have considered buying a second home to rent, but I have some moral qualms about exacerbating the housing crisis where I live. Furthermore, the stress of tenants isn't something I really want to deal with.
Everyone here has great points about maximizing returns, and I know I will have less money in the long run because of my decision. With that in mind, I am investing about half of my old mortgage payment, and the rest goes to the family vacation fund.
Edit: having said that the difference is not that large (3-4% for the 30 year note, vs. 5-10% for the market return). Also, while I didn't pay off my mortgage, I probably won't put even more money where my mouth is and refinance in order to invest the cash-out into a fund.
If you have the money to pay off your mortgage, why not buy a second house with it and rent that out? Let someone pay your mortgage while you get the appreciation? Or invest it in something else?
If you do the math, renting almost always comes out ahead of owning, as long as you invest the difference in something that gains in value.
The main reason to own is for psychological reasons. It's great if you have kids and want a place for them, or yourself, to call home.
Yes, there's always the risk of a downturn or recession/depression that ruins that plan. And beyond that, there is often a great psychological benefit to being debt-free, even if that's not the best financial decision.
Index funds don't always rise and property values sometimes fall. Interest rates are rarely this low. Leverage multiplies both the upside and the downside.
While this is true in the short term, except for very rare exceptions, you'd be hard pressed to find a property in the United States that is worth less now than 30 years ago (which is the standard length of a mortgage). I don't know about other countries, but I suspect it's the same in any modern economy. Land is scarce, and no one is making more of it.