NIMBYs. Builders want to build, but the people who already own won't let them in most cases. I live in Cupertino. They want to build 2000 units next to my house. Most of neighbors are against it because it will "change the character of the neighborhood". That is true, it will. But I'll be the farmers who lived here in the 1960s said the same thing when all of our houses were built as the farmers sold their land.

The main difference this time is that the current residents can't get rich selling their land because the developers want to build up, not out.

CA tried to solve this with SB 827, which would force upzoning near transit. Sadly, it was poorly written and failed to pass, but it was a good idea. It would have forced pretty much all of San Francisco to allow building medium size buildings in place of existing single family homes. And a lot of the rest of the Bay Area too.

I've seen regulations brought up as a reason why there isn't much new construction, but other states with similar building code/environmental regulations don't seem to have the same problems.

The exception to this IMO are zoning regulations. When those limit high density housing, eg condos/apartments, they can serve to substantially increase home prices. But, that's assuming builders are willing to build enough units fast enough to actually reduce unit prices, and they've consistently shown they will not do that.

Prop 13 could contribute somewhat too, but I imagine it's effect is also much smaller than private equity and builders limiting supply.