That's exactly the problem. Google is notorious for providing horrible customer support, so why some people get personalized help, while the rest of us are stuck with the uncaring robots?
Giving occasional personalized help is a great way to avoid a total black hole while still creating an escalation path. It's like having EXPLAIN QUERY stack traces on NewRelic. You don't need a deep trace of every transaction; you need a deep trace from a few representative squeaky wheels, and you use that information to improve the underlying self-serve process. This is something Google, with their "1/3 of the company parses logs", excels at.
You'll note neither Matt nor Pierre offered to do anything special for PG. They didn't say "Oh! I love HN, and we want it to succeed; I'll go push out a manual override that makes sure we use only the one crawler you unblocked." They gave him exactly the info he'd get from Webmaster Tools plus general knowledge, and told him how to use Webmaster Tools to fix it. Maybe they'll go figure out how to implement that "you're blocked!" hover-over in SERPs, but again, that would benefit everyone, not just HN. There's no special treatment here.
[1] http://blog.wordtothewise.com/2008/04/troubleshooting-a-post...
The point is if you can build relationships with Google employees rather than with Google itself then you too can get this sort of amazing help.
There's nothing sinister or crazy about this. Rather than being one in a million, become one in a circle of friends or acquaintances.
PG knows his shit. He knows how to block crawlers, he knows how get SEO done, he's no dummy. "So what?" you say. Well I work in web development and I meet with people all the time and explain to them how to run a site and get rankings up, etc. I tell them I'll give them the tools, show them how to use them and even leave detailed manuals on how to work everything. These people are so into it at our first meeting and they're excited to get into it and work this stuff.
Then the job is done, the tools are handed over, and the lessons are learned. What do they do? Nothing. Then they do nothing. Then even more nothing. The site is never updated and they do no maintenance whatsoever and eventually it becomes outdated and dead for all intents and purposes. A year later they want to know why they're not ranking in Google. I ask if they've checked or even once used any of the tools I gave them and the answer is always no. Then they demand I fix it. I kindly remind them they're the ones who wanted to have Larry in accounting run the site because he's "good at computers" and to save money by not hiring an IT guy.
So the point I'm making is that I'm sure there are a ton of people out there who are trying to get customer service without trying to help themselves or using the resources readily available. I think it's great that Google is showing they're willing to help those who help themselves.
I think google isn't treating everyone equally but they are treating people fairly. There's a difference. I'd prefer fairness. Not all "customers" are created equal so you compensate by treating them with fairness instead.
Apparently he doesn't use Google Webmaster Tools, though.
With regard to this issue doing the above is not evidence of knowing your shit in the area in question.
Being bold is an important aspect of getting to know your shit.
Also, besides this, this action has gotten us talking about how Google throttles crawling, and this discussion has further reduced my opinion of the World's Largest Adware Vendor.
Now if I were to ban Google searchbots based on what PG is saying I would simply limit them to the first page or two of listing. I think this can be done in a robots.txt?
