Here are a few broad generalizations:
The "tech gradient" progresses from component/hardware manufacturers in San Jose to data centers in Santa Clara to software developers around Palo Alto, to designers in San Francisco (or "The City," as SF hipsters, often call it). Of course, there are many exceptions to this rule of thumb.
Many big-name venture capitalist firms are located on Sand Hill Road in Menlo Park, situated in the middle of the Peninsula. With a BMW (also known as Basic Marin Wheels, after Marin County, to the north of The City), Sand Hill is easily accessed from 280, the quicker of the two Peninsula freeways. Some of the more interesting VC firms are actually located elsewhere: downtown SF, Oakland, warehouses in San Bruno, by SFO, Mountain View (in the case of Y Combinator)... Then again, given that you're posting this on Hacker News, vulture capitalists may not be your primary concern.
More and more pedestrian pockets can be found outside of SF. A few that come quickly to my mind are:
- downtown Mountain View (Castro St., with an Asian market)
- California Ave. in Palo Alto (with a co-op)
- downtown San Jose (actually getting kinda lively)
- downtown San Mateo (with a great Japanese grocery store)
All of these places happen to be on the Caltrain line, which runs from SF to San Jose (and points further south). Find a Caltrain stop and you'll find a cluster of start-ups and condos and cafes.
Much as I try to bike and walk everywhere, I've had a number of around-the-bay days, that begin in an office in Mountain View, progress to a meeting with designers in the East Bay, and conclude with an investor pitch in The City. With demo computers to schlep and a schedule to keep, unfortunately, I just gotta go with a car. But perhaps that's what Zipcar is for.
Finally, let me put in a pitch for Berkeley, where I live now. It's a little apart from the SiliValley action, which can actually be a benefit in the personal sanity dept. Easy biking, good food, nice sailing, lively city, a world-class university...
Good luck!