This is why you never invest more than you are willing to lose. If you do, and the venture fails, the blame for your reduced quality of life rest not on the failed founder, but on the imprudent investor.
The nice thing about SV is not just the big pool of investors, but the big pool of prudent investors who don't behave foolishly.