It matters on when. Austin is one of the fastest growing cities in the US for this exact reason. Its having a HUGE impact on the price of housing. What I bought in 2009 is now worth double (good for me, bad for new buyers).
We have shit transportation infrastructure & the housing market is (for us) insane.