Ask HN: coding challenges as part of interviews: yes or no?

My experience:

Successfully pasing a code challenge is not a good predictor of future success. However, failing to pass a code challenge is a very good predictor of future failure. Furthermore, making the test harder does not make it predict future success well.

Therefore, I belong to the "FizzBuzz" school: Make the test ridiculously simple. It should take no more than fifteen minutes to complete. If the person gets it obviously right, throw the result away and move on to more important quetsions. Do not try to deduce a lot of malarky about their programming style or ability from an obviously contrived problem.

For example, writing:

a = 5; b = 10

Does not mean they don't know Ruby well, and writing:

a, b = 5, 10

Does not mean they are a Ruby expert. And writing:

fsquare = lambda { |x| x * x }

Does not mean they are a closet Bipolar Lisp Programmer. If the program works, it's a pass, move on. If there is a syntax error or some such, who cares, move on.

But if they struggle... You need to investigate the reason for their difficulty with a simple problem.

Trick questions, or questions with some kind of gottcha have no place in interviews, However, simple, straightforward programming exercises are essential. I've seen people who claim to have X years of industry experience who can talk a good game, but literally can't write a switch statement without reference to the internet.

I also like to ask people to peer review some code with both obvious and more subtle errors in it. That's a pretty important skill to me too. Anyway, ask them to code some fairly simple algorithm and then critique their own work. If there are several ways to solve the same problem, some more efficient than others, all the better.

A trivial coding problem is essential. I've seen people with 10 years of experience in XML#Java.NETEnterprise2012 who don't understand conditionals, loops, or boolean expressions.

For phone screens, I find Steve Yegge's guide a useful starting point: http://steve.yegge.googlepages.com/five-essential-phone-scre...

I'm not a fan of asking candidates to reverse a string or to implement atoi or a binary search, because those have corner cases which, under interview pressure, might be too much to ask for. I just ask for an implementation of a gcd (greatest common divisor) function, in any language. I always explain the problem in detail. Only one guy (a PhD in physics) ever gave me Euclid's algorithm (section 1.1 in Knuth's AOCP Volume 1), so I don't expect it. I do expect a trivial loop, and most candidates take 15-20 minutes to come up with it, and need lots of hints.

As far as I'm concerned, anyone who can't come up with a trivial loop (or recursive or tail recursive function) for something like this shouldn't be programming for a living. Maybe "developing enterprise solutions," but never programming.

This reminds me of a group of main frame programmers I knew who had been forced to become Perl web developers. About a month later they were asked to hire another Perl programmer and one of their interview questions was what's the difference between 'chomp' and 'chop' (chomp removes the final character of a string if it's a newline, chop does it regardless).

They were shocked that the interviewee didn't know the answer because they considered it basic knowledge, "it's in the first chapter for gosh sake." The first chapter in this case covered differences between Perl 4 and Perl 5.

In Perl 5, nobody uses 'chop', they all use 'chomp.' Many experienced Perl programmers might even have forgotten that 'chop' existed.

And that's my major problem with gotcha questions, they tend to be pretty far from measuring if a person can actually write productive code.

Programming challenges sort the wheat from the chaff - in particular those who cannot code at all, but beyond that I've rarely found them predictive of future performance. Much more important that you hire a good fit for your company. An unhappy or toxic coder can bring a startup to its knees - hire slow, fire fast and all that.

I like them as long as pseudo-code is acceptable. I don't really care how good someone's memory is with regards to language syntax; I care about whether or not they can solve problems and think them through.

/agree with most of the people here. Coding questions have a purpose.

You need to consider what you are trying to determine by asking any question. Really, what is it about the candidate you want to know? After pinning down precisely what you want to know then tailor a question to figure this out with as little peripheral stuff as possible.

If you want to know if someone can generate some code in their preferred language on the spot, fine. Pick a very simple problem, like 'Write a program to display numbers in the fibinocci sequence', and explain the fib seq to them if they don't know it. I wouldn't even count points off if they make any kind of mistake that a compiler would catch as long as they know how to fix it. Have them do it on the white board.

This kind of Q isn't designed to figure out how much they know of 10 different languages. It's designed to see if they can put some code down in their preferred language and have it do something useful.

The biggest constraint in an interview isn't the honesty of the candidate, or even their knowledge; it's the N hour time limit. Your goal as the interviewer is to squeeze as much information about who the candidate really is in the time allowed.

I like to believe that I can get a pretty good understanding of someone's technical abilities by asking thorough questions about his previous experience, and maybe ask them to explain a couple of basic patterns or programming concepts to me. I remember interviewing a senior java developer who claimed 5+ years of experience and had no idea what a Singleton was. What made it worse was that even after explaining it to her, she was unable to figure out how to implement one. Developers asked the strangest questions when interviewing other developers. I worked with a guy once who liked to ask his interviewees what the decimal value of 0xFE is. One candidate didn't know the answer so my coworker didn't think he could possibly be a good software engineer. I admit that (having done assembly coding in school) it seems odd that any software engineer wouldn't be able to figure out what FE is. But I still thought it was a bad way of testing technical skills. Would you rule out a candidate who doesn't know 0xFE ?

I think taking some time to sit down and _read_ some code is one of the best and least used techniques during an interview. I think it's more of an indicator for job performance than writing some green-field code. After all, on the job, more time is spent reading and understanding code than writing code with no preexisting context.

I think it depends on the position and responsibilities the person will have once hired. If the programmer will be coding directly from specs without any room for "interpretation" then a coding challenge may be very applicable. If you are looking for a programmer that also understands your business and will be required to make decisions and think on their feet, the value of a coding challenge diminishes.

It also depends on the time frame of the position. If you are looking to hire a programmer on a per project basis, then a solid understanding of the development language is critical. However, if you are hiring an developer for the long term, it's better to hire a smart and agile person that can contribute to the overall business as well -- which is hard to measure from a coding challenge.

I also worked for a company once who had several enthousiastic , seemingly bright people with somewhat limited experience come in for interviews, but they didn't make the cut because they didn't pass some coding challenges. The guy who did get hired went to a good school and was a good coder, but was way too arrogant, difficult to work with and repeatedly refused to do what his manager asked him to work on. And how did that guy interview other candidates? He screened them by their education, and when they would come in he would give them one programming challenge without asking any other questions. Nobody seemed to realize that there were big problems with the company's hiring strategies and that developers themselves were largely to blame for that.

I've always found that asking a reasonable numerical solving question where you can easily figure out the answer in your head is a good starting point to gauge programming/algorithm ability. If someone knows the solution they are trying to get at, they can focus on the program itself, as opposed to worrying about solving the problem and coming up with the algorithm. Ideally there are a few ways to solve the problem, brute-force methods or more efficient ways. It can be very difficult to test a programmer's ability in an interview, you need to focus more on seeing how the candidate thinks, how they approach problems, and whether they can admit when they do not know something.

I use a simple coding challenge as a weeder question. It requires basic use of recursion (without explicitly saying so) -- no math, no big-O, no gotchas, just writing a simple recursive function to process a small tree. It takes decent coders a few minutes to complete. Less-than-decent coders really struggle with it.

This is only a wheat/chaff separator, though. If possible, I like to see code from actual projects they've worked on to get a feel for the bigger picture (tip: open source your personal projects and obsess a bit over the code, so you have something to show off). Plus there's all the other non-code aspects to the interview.

If you have the sense from whiteboard discussions that the candidate understands computer science (like gets recursion) but you want to make sure they can code, why not choose a subset of a problem that you (your company) needs to solve?

They're a necessity. For the simple reason that too many people lie on their CV, no matter how good their CV looks you have no-idea how much of it is accurate.

Obviously there are cases where you wouldn't use it, but generally they only apply when you know someone's capable of coding anyway (i.e. due to an indisputable reputation, experience working with them, etc.).

However as a predictive measure it's fairly limited in that it's a binary indicator. If someone fails then there's a fair chance they're a terrible developer, if someone passes there's a fair chance they're not a terrible developer.

Necessary but not sufficient.

I always give a fairly simple problem, pencil and paper, a soda, and leave them alone for 15 to 30 minutes. Before I leave the room, I make sure they clearly understand what's expected.

What they do isn't important. Our discussion about what they did is. I don't care what syntax or mannerisms they use; I do care if they understand what was needed and what they did well enough to explain what they did and why they did it.

Almost every time some good follow-up questions and segues ensue. What about this? What about that? I've learned more about people in the discussion after the problem than from anything else.

If I were interviewing someone, I would be more likely to test someone's general problem solving skills, questions that illustrate how a candidate thinks and approaches a challenge. This of course becomes more subjective, but can really help identify creative talent. Who knows, you might even learn something too. This is probably a big part of the puzzle type questions that companies like Microsoft and Google seem to favor.

Agree with jamess, but additionally, I think interviewers that don't ask coding questions might be an indication of something negative at the company. Basic coding ability has to be one of those checkbox questions that determines your eligibility for the position, and if the interviewer doesn't ask anything technocal, it makes me wonder what kind of dumbasses I might end up working with.

Writing code is artwork. You can ask the artist about the specifics of his tools, but you will never grasp his artistic ability until you see the completed artwork. I just ask some general questions about the tools. The tools for code writing are: starting, constants, variables, addressing, comparing, branching and ending.

From the employees point of view: I would have no problem to do a coding test as part of an interview.