Its also how wealth is transferred up. Because the investor pays gains rates, can do like for like transactions, etc, and effectively grows their wealth faster than the non investor. This means non investors need to continuously compete with investors but where investors are capturing an ever larger share of the wealth.
Eventually if supply keeps up that will change, and longer still the wealth may trickle back down. But imho it would be faster to focus on avoiding the concentration in the first place through SFH as investments, via simple tax policy that discourages mom and pop investors from purchasing extra homes.