1. Finding decent people that are both technically competent and a good culture fit is bloody tough and can take a lot longer than you'd ever expect.
2. Interviewing/screening these people is simple. Don't make it more difficult than it needs to be.
You've done the hard part and you're confident the person/people you've found are worth at least a couple of hours of your time for an interview so why waste that time by applying a boilerplate interview process that every other company uses?
Give the candidate a realistic technical challenge to complete in a realistic timeframe, in an environment that will be indicative of the environment they would expect to work in should they succeed in getting the job.
If you're happy with the technical results, spend at least an hour with them having an actual conversation. Don't sit there throwing stock interview questions at them, don't try and nitpick on their CV. Just talk to them. Get a feel for their personality, the things that motivate them, the things that annoy them, the things that inspire them, etc.
1. I was repeatedly made to feel welcome, comfortable and calm as I possibly could be by everyone I met.
2. I was given technical questions that were appropriate and highly correlated with what I would be doing on a daily basis.
3. I spoke directly with two founders who, despite having my resume, opted for talking to me about my background, my goals, and offered answers to any questions I could possibly think of.
4. I was interviewed by four people total, and only one at a time.
It was honestly the most enjoyable interview I've ever had, despite the fact that I was initially nervous and didn't know what to expect, and it took the better part of a day. It was literally fun - if nothing else, this experience has given me a model for how I would interview candidates in the future if I am ever a hiring manager.
I wish every company operated this way - on some level, I think a lot of people who interview candidates don't know what they should look for, so they fall back on the "legendary battery" style of asking a suite of questions that only a recent 4.0 MIT CS graduate could answer confidently (and who could totally fizz out when asked to do something that isn't in Intro to Algorithms).
The problem is that a realistic challenge involves figuring out a realistic system. Realistic systems tend to be large, complicated, hairy. They tend to involve a lot of domain knowledge. They tend to have subtle considerations.
Working with a realistic system does not generally take a realistic amount of time. Not for an interview, at least.
I became a chef, where the interview could be moved to having them watch me make a dish or show knife skills. Then out of interest, I went through a biomedical engineering degree and am back into the tech field.
Still, due to my negative experiences with the filtering process, I have stayed in the realm of freelance and contracting. I would jump back into a tech job if interviews, like the above, were more common.
Thanks.
What exactly are you trying to determine from this? The problem with such open-ended types of questions is that all objectivity is lost. You end up picking people who mirror your own motivations, annoyances, etc. Unless you are sure your question will produce actual signal in an objective manner, you shouldn't ask it. You are just biasing the process in favor of people that will flatter your own ego.
[1] https://news.ycombinator.com/item?id=5227923 (this earlier Hacker News comment gives full references for the statements in this comment)
Making decisions about managing technical debt, adding architecturally-significant changes, balancing good OOP with responsiveness, knowing the difference between future-proofing and conscientious coding -- all of those are both crucial (in many cases the most crucial) to day-to-day work, and also so highly context-specific that those decision-making traits are nearly impossible to identify during a technical exercise.
So for coding challenges, that leaves short-term tactical/analytic/algorithmic exercises, which in (anecdotally) 95% percent of cases cannot begin to approach a 'work-sample' environment. I've yet to encounter a technical challenge that would tell me much more about a candidate than basically how fluent they are with their tools, how well they know syntax and some general design principles, and what, for instance, their TDD (or lack of) workflow is like. Probably some insight into line-level analytic and algorithmic ability.
All of that is helpful, but -- Trust Me Here!! -- can also be very deceiving. The same coders that can knock those challenges out of the park can also be highly-proficient Debt Machines, all the more destructive because of their special genius for cranking out architecturally suspect code at a breathtaking rate.
To get into a real 'work-context' flow of a large app requires weeks, sometimes months, and only then can you get full perspective on how a given coder is going to contribute to your team on an ongoing basis. To get a feel for what that will look like in an interview, I've found I have to pretty much rely on the candidate's past projects, and informal conversations around larger architectural and OO principles.
[1]http://www.psychologie.uni-mannheim.de/cip/Tut/seminare_witt...
The validity coefficient provided by Roth and Bobko is likely more accurate. That isn't to diminish their value as they are still valuable, but the aren't the cure-all we'd like them to be. They do continue to show promise in reduced adverse impact though, which is great (note: the full article is behind a paywall - what is the HN-approved method of sharing the information?):
[2]http://onlinelibrary.wiley.com/doi/10.1111/j.1468-2389.2010....
That is for gender. With regards to ethnicity, the evidence isn't quite as optimistic yet. Like other predictors including cognitive ability tests, if they are showing notable adverse impact you may be in trouble Like other predictors including cognitive ability tests, if they are showing adverse impact you may be in trouble despite their validity.
[3]http://onlinelibrary.wiley.com/doi/10.1111/j.1744-6570.2008....
It's a problem with lots of predictors, though scope of the problem varies. There is work being done all the time, even in the most reliably stalwart predictor, the cognitive ability test:
[4}http://psycnet.apa.org/journals/apl/92/3/794/
Anyone interested in the great "diversity-validity dilemma" can check out this link for more information, though there's always progress. It's a great article.
[5]http://onlinelibrary.wiley.com/doi/10.1111/j.1744-6570.2008....
For my money I endorse integrity tests as a part of the solution. Decent validity, including incremental validity over cognitive ability due to a low correlation between the two, and small sub-group differences.
[6]http://onlinelibrary.wiley.com/doi/10.1111/j.1744-6570.2007....
Having said all that, I imagine the efficacy of work samples is moderated by the type of work, and I'd have to believe they are more amenable to demonstrations of technical skill like coding (I don't know of any references for this now, but I'll look later). Coding-related jobs would be nice because it would be possible to blindly judge on the output as well, and in programming-type jobs it would be much easier and cost-efficient to test large numbers of applicants than it would for many other jobs. Cost and ease of large-scale administrations are their big problems, so overcoming those would be gravy. I don't know how subgroup differences are impacted though.
What I've done should be proof enough. I don't want to jump through a bunch of arbitrary hoops so that I can get an incrementally higher paying job (maybe) and devote my life to working on someone else's vision. Gee, can I?
I already have a job, and it might be different if I didn't. But maybe not.
It really isn't. It may be indicative of your technical ability however your past experience doesn't give me any indication if you'd be a good culture fit.
Just like in social situations you can judge anyone pretty well in the first 5 minutes of a phone conversation, see if they will "click" with your team - hell you can almost do that just by looking at their open source (how they think/design). I don't need a high school coding exam and a 2 hour panel interview to see if someone is a "cultural fit".
The trouble is that, well, I hate to be like Kant but a lot of recruiters do just seem to have conversations that are very... spanish-inquisition-y... with virtually none of your personality there.
Like, when have I ever had a joke with them? Or an interesting discussion? When has a recruiter ever asked me about the people I work with/my friends? Or spent any real time talking with me my about my hobbies?
Hard to bring your personality out when people are asking you things like - "Would you consider yourself competitive?"
There's an underlying thing there where they're trying to guess at qualities they think contribute to behaviours they want in a very structured way rather than having a real conversation. And I'm sure we both know that people are more diverse than that, you can have lots of reasons for behaving in a particular way - not all of which the other person's going to be able to guess about.
It just feels really awkward, you know? You want to know how I'll fit in with your culture, invite me out for a glass of wine or something and we'll find something to laugh about. You want to get fake-happy-happy me then stick me in a horribly formal situation.
That said, I'm sure there are recruiters that treat you as human - and the more of them that are out there the better.
What does that mean, how is it determined via an interview and how is it weighted against all the other factors that make up a good hire?
Did I say it was the only thing? No. I said that the burden of proof shouldn't require three phone screens, a multi-hour (7 - 8+?) coding test, and then a 5 hour on-site interview. Companies are taking time out of people's lives to string them along on a hiring process that could last up to a month and a half, and still could result in absolutely no offer.
Second, I'm not merely referring to the list of things I've done. Like many people, I have public projects online that you can download and use, and in many cases, these projects have source trees that are publically available. Yet, most of the time, companies don't even care enough to do the background research of downloading the application and using it, or looking at the source of these applications to see if they are any good. I've submitted examples of work I've done and when I got to the on-site interview, the interviewer had no idea what I was talking about - hadn't used the application, hadn't seen the source, and in one case (where the application was actually quite popular) had never even heard of it, despite the fact that I'd mentioned it in at least two of my interviews.
Companies should be respectful of people's time. They shouldn't make the burden of proof so high that only the most desperate would ever agree to jump through these hoops knowing what they are in advance. I shouldn't have to complete a 10 hour programming test and then spend 5 hours proving to you on a white board that I can code. At what point do you say that my knowledge is general enough that I can pretty much approach and come to a meaningful solution for any problem that you can throw at me ? The combination of a short, in-person interview and a review of my work history and my references should be sufficient to get a grasp of my work skills and this vague concept of 'cultural fit' which seems to amount to, "Are you nice and respectful and a team player?"
Any thing beyond that is disrespectful to the candidate, especially if you are trying to recruit them away from another full-time position.
I don't think anybody reasonable is going to suggest that there's a single criteria on which a hiring decision should be made, but if you don't think that what you've accomplished should be pretty high on the list, finding candidates with an ability to create real business value is probably going to be much harder.
Who would you rather hire: a person with a track record of delivering working solutions or a person who can code on a whiteboard?
That, and I wonder if there's a certain fraternity-like hazing element. One of our HR people recently brought up the possibility of removing one of our hiring steps (a take home programming assignment), and I found myself irrationally against the idea. I even joked that I wanted new candidates to suffer just like I did. On reflection, that might not have been a joke... Not particularly proud of it, but part of me resists changing the process because I did really well at it.
Anyway, good luck with that.
So when was the last time your preferred language's sort function magically disappeared and you had to come up with your own?
The interviewing process is criticized so frequently because companies focus on petty questions that are unrelated to a candidate's ability to do the job.
No offense intended to you. I doubt you meant it like this, but I got the same impression.
FYI, responses like this serve only to confirm the initial impression that you're insufferable.
The article was peevish, but I identify with the author's viewpoint. I suggest they interview with smaller companies, where for instance you can meet everyone and chat more informally. Like a startup.
1) they were hired before the company started doing it
2) they new someone going in so they got hired without as much rigmarole.
3) they started as a contractor/freelancer and were converted to employee without interviewing
4) they came to the company through an acquisition
It seems like if you're not good at these annoying pre-offer activities that you're better off taking one of the above routes into a new job rather than trying to play these games.
[edit formatting]
And if we are talking about major corps, it would require lots of investment in terms of internal training, would take a while, in other words - it's expensive. And they still get a lot of top quality candidates, so why bother?
I think you are on to something here. People want to do the safe, easy, and known thing. I cannot imagine most people, when finding out a poor employee was hired would blame the interviewer or the interview system. They assume that those are fine, because "Joel, Yegge, Google... everybody does them" and so they become a mental blind spot. The focus falls on the bad employee, "wow, he seemed so good, he must be a con artist or something".
>And they still get a lot of top quality candidates, so why bother?
This, I do not believe. By definition, most developers are not "top quality", rather just fit in just well enough to stay out of the "fire zone". From what I have seen, unless by "top quality" you mean "best development value for your buck", most developers at most shops are far below "top quality". Most are either overly conservative or overly aggressive, and have skills just barely good enough to do their job, which is good, because if they were truly the "top quality" they would tire quickly of your work and leave. The only way someone gets to be even close to "top quality" is by constantly doing new things, constantly pushing themselves, constantly getting better. Those traits _sound_ good till you consider what then will happen when they master the job you want them to do and get bored. That is what I mean by "best development value for your buck". Chances are, most development shops want most developers to be just barely good enough to get the work done in an arbitrary time with an arbitrary level of quality. Like the old joke goes, anyone coding slower/poorer/less than me is "a lazy idiot!", and anyone coding faster/better than me is "an architecture astronaut stuck up jerk!". Hiring becomes, "are you within an acceptably narrow band of skill that will all share?"
So, companies get what they hire for. Chances are, they get mediocre programmers, because that is what they actually _need_. They don't need language-writing, development world colossi striding about their office, designing new paradigms for their slightly altered CRUD app. Most companies wouldn't know what to do with such developers if they had them. Heck, even Google apparently didn't know what to do with Guido. Instead most companies need someone happy sitting down to add a new field to this page, or make this checkbox work on this popup, or fix this bug that only shows up in IE7. Someone for whom that is just challenging enough to be interesting, but not impossible. Apparently, programmer quizzo is the cheapest/safest way to ascertain that.
This is a great point. I've noticed that culture tends to be pervasive. The good experiences I've had with companies started from first contact. So have the bad experiences.
In fact, a candidate might have experience in something completely different but maybe just as impressive - publishing a paper on an optimization for an algorithm, creating a programming language, submitting Linux kernel patches, etc. How would one differentiate a candidate like that with someone with "a list of public repositories on GitHub as long as your arm"?
In fact, that's kinda the point. Each candidate should be evaluated on the unique experience they bring to the table. Trying to create the illusion of a level playing field by standardizing on generic code challenges in interview exercises is just that, an illusion.
As for the original article... if a company puts you through a long interview process, you should take advantage of it to learn about them as well, and see if it's where you want to work. Don't just be a replaceable part.
The game changes however if you need an expert consultant/contractor/freelancer who can solve a very specific problem with a very short period of time. For those, go hard on the specific tech. questions.
And, yet, our demand for developers with experience surpassing the average co-op or college grad doesn't change. It can be hard to make much progress with a team of mostly younger developers when every technical discussion someone is asking, "what is FTP", or "I read that GET is the only safe way to pass values", etc. Sometimes, it _really_ pays off to just hire someone who has already figured out a certain level of the fundamentals and can hit the ground running into more advanced walls like lack of business domain knowledge.
1. Write some piece of code that you think is cool. It doesn't have to originally be yours, and if it would take too long to recreate exactly, write it out in pseudo code and explain how it works and why you chose it.
2. (Slightly more common, and I've asked this before) We are having Problem A, and it would be within the scope of your responsibilities. How would you approach it?
the interviewer asks follow up questions to both and hopefully learns something in the process.
If your problem with the question is trying to think of a response that the interviewer would want to hear, then you aren't approaching the question correctly and/or neither is the interviewer.
He threw out 1:n interviews being bad as if it were the norm.
I also have no idea what he meant by this:
"All of this for a position that will involve working remotely (you are working remotely, aren't you?)."
Stopped reading after that because it didn't sound like he and I were in the same industry at all.
Something so simple became more difficult under the pressure of someone watching you the entire time.
It's also nothing I've ever even thought of building nor something I would ever be asked to build in a real work environment.
I have stellar reviews from my former employers. I know I don't suck, but hell if that didn't shake me to the core.
Maybe I'm just venting.
What filter? I didn't see him mention the filter he's using. Maybe he means he rejects a company unless it has an adaptive interview. But, how can you do that unless you take the interview?
