Yeah, but SFH aren't the only housing type appreciating. This theory doesn't really hold when you consider both Condos, inner city locales, rural locales etc have also experienced rapid price appreciation.
Rental or not doesn't really matter. Fundamentally people just need a place to live, and depending on locale will make the tradeoff between renting and buying. If rental stock doubles overnight, housing prices would decline too, as cost of a mortgage becomes relatively less appealing. In this sense it's fungible.
But it's true there may be some subset of people who would only buy SFH regardless of price
