I agree with you. In my opinion Silicon Valley is a great place for the following types of software engineers:
1. Those who work for FAANGs.
2. Those who work for non-FAANG companies that are generally well-known and have compensation packages that are competitive with FAANG, such as Uber, AirBNB, Pandora, Twitter, etc.
3. Those who work for VC-funded startups, given the strength of Silicon Valley's VC ecosystem.
But what about people working for more traditional tech companies such as HP, IBM, Intel, Cisco, and the like? What I've found is that the compensation packages are usually not at the same level as non-enterprise tech companies. When I got a job offer to work for a FAANG after getting laid off from HPE, while my new salary was a modest 10% raise, what made a big difference was the RSU grants. (Unfortunately things didn't work out, but that's another story.) This difference in compensation matters, especially when it comes to milestones such as starting a family and purchasing a house. It's much easier to purchase a home within commute distance to Silicon Valley when you have six figures worth of stock to sell (which is possible at some FAANG companies for employees who spend a few years at the company) than when you don't (which is the reality for many non-FAANG engineers in Silicon Valley).
For those working for more traditional enterprise companies, their compensation would go much further in places like Houston, Sacramento, or even some Seattle suburbs (think Kent or Lynwood, not Bellevue or Redmond), where they can comfortably afford a house, as opposed to Silicon Valley where purchasing in even Livermore or Gilroy may be stretch goals. I'm back at an enterprise company that announced months ago that we'll be working primarily from home even after COVID-19 ends, only coming to the office infrequently for meetings. While owning a house in Silicon Valley seems like an impossible dream for me, I could decide to move to a place like Sacramento, where I can afford to buy a house on my salary and still be close enough to Silicon Valley to drive to infrequent meetings.