It also gives you an idea of what the company focuses on - delivering working software. I'm not saying companies that don't do work samples don't focus on that but as a candidate it makes you wonder if playing games is part of the job. You gotta talk about real code, look at real code and discuss it.
I have also been a manager and sat on the other side of the hiring table and it is hard to judge the technical chops of people. We used quizzes of deep technical items that should have been known by someone if they had done the work before (e.g. MTU settings for a db cluster) and that helped weed out people but it really wouldn't give a sense of if they were a 100% good fit, you need other ways to check that too.
I like the analogy of developers to chefs and cooking. Programming puzzles is like judging a chef on cute tricks to slice an onion. Would you hire a chef based on their onion slicing tricks or their ability to cook an amazing meal?
All of that said, I have to say spreedly’s work sample was the most realistic. During the process I did feel like there was a purpose to what I was doing, which is great for someone like me who has a hard time doing “puzzles” and the like because I see no utility in it.
Do I think blind hiring works? It could, but not the way it's being utilized. When I first heard of blind hiring, I thought this is great. No bias, no bs, just code. I've been on the hiring side of the table before and 90% of the applicants were full of it in regards to what they put on their resume vs what they actually knew. The worst part is after you spend the whole day interviewing worthless candidates you have all your regular work that has piled up during the process that must be done. A lot of time and energy gets expended in that process and wasted. I truly understand both sides. As a tool to filter out "those people", it's great. However, blind hiring seems like it being used to try and find the perfect candidates. Perfect candidates do not exist; just as perfect humans do not exist. Great test takers do exist though. Personally, I have never correlated good test takers with good employees or good programmers. In fact, it’s always seemed to be the opposite from my experience.
So, what’s a better way to skin this cat? I honestly don’t know, but if I think of it I’ll be sure to make a proprietary app and sell it to all the companies I’ve done blind samples for : )This is great, Peter, thank you for this perspective!
If you walked away with the impression that there were gotchas, then we have failed. While we do have specific criteria we're looking for (as we're trying to make the grading of each work sample as objective and consistent as possible), I wouldn't call those items "gotchas". It's also not the type of thing where if you miss one small item, you're disqualified. The grading allows for several missed items and is only looking for some total of all allowed points (across 33 different criteria). So while some of the items you didn't get credit for might have appeared trivial, or like a gotcha, there should be enough allowance in the process to not let one or two misses scuttle the whole effort.
> Perfect candidates do not exist; just as perfect humans do not exist. Great test takers do exist though. Personally, I have never correlated good test takers with good employees or good programmers
We are totally with you re: perfect candidates. Work samples are only one part of the hiring process after all, and are meant to ensure some base level of technical ability. Others parts of the process, such as our structured interviews, are meant to flesh out some of the nuances I believe you're referring to here. We're not looking for a perfect candidate, but we are looking to understand what makes up each candidate to know if their strengths are a fit for Spreedly and if we can live with the weaknesses (and vice versa for the candidate evaluating Spreedly!).
We really do appreciate your effort, both in writing up your thoughts here, and in applying in the first place. This is how we get better at this kinda of thing!
Oh, and remember... next Friday... is Hawaiian shirt day. So, you know, if you want to, go ahead and wear a Hawaiian shirt and jeans.