It is not an accurate measure of candidate technical skill, hence the rise of take-home tests. (the submission reminds me of the blog post "I Won't Hire You If You Can't Pass These Programming Questions Because Otherwise You Suck" that was written awhile ago but I can no longer find. That post backfired hilariously because author provided the wrong solution for one of the questions.)
After the hour was up, my code was reviewed by six people while I interviewed with the DBA who took joy in pointing out my admitted deficiencies with SQL. I then had another hour discussing the code with the six reviewers.
The program worked on the sample data I was given. I was later told that the other candidate's programs also worked on the sample data. Except, mine was the only one which actually ran to completion with their real data sets. I was also the only interviewee without a formal CS "education".
I got an offer which I ended up not taking due to location considerations.
If you are going to ask someone to write code, give them a computer and some quiet time. You can grill them about the thinking process after they came up with something. Having someone watch over one's shoulder every time one puts a pen to the stupid whiteboard will eliminate a lot more candidates for the wrong reasons than one ought to consider desirable.
The author says that the candidate's thought process behind arriving at the solution provides insight into the candidate's skill.
There's no reason to implement your own binary tree, but knowing how to devise algorithms to accomplish a given task is a necessary competency.
I don't think that being able to describe an algorithm you haven't had to implement in more than decade (at least, that would be the case for me, college is starting to be a long time ago for me).
Honestly right now, without looking up the answer, I probably can't reverse a binary tree on a whiteboard. I could, however, do it by test driving a solution with a real computer (I haven't flexed those muscles in a long time, but test driving would help me dig into the recesses of my brain).
Which skill set is more useful for production quality software? I can say for sure that it's not white boarding.
Agreed. Why not use a realistic problem and setting?
Furthermore, the business value of a programmer is rarely how she writes code in isolation. It's how she writes code on a team. I'd take a B-level programmer who is excellent at writing clean, readable, simple code over someone who is a savant at white-board problems.
That's a blanket statement. If we find one exception to this rule, then we can say that it's 100% false (since its truthfulness is binary -- it's either true or false).
To find one exception, we only need to find a single "good" piece of software whose author can't write 100% of the dependencies, including: (probably) C/C++ compilers, FS drivers, low-level network libraries, and the widely-used monster that is SSH.
Trees are the single most important data structure in computer science. Just
about everything you do in your programming career will be related to trees.
One thing often forgotten with hashmaps is that they aren't actually O(1); they're O(k), where k is the length of the key, and often need to examine the entire key to derive a hashcode. This oftentimes makes them significantly slower than a binary search tree when the size of the key is large compared to the size of the container.
As always, measure before optimizing. YMMV.
> I just couldn't resist a chance to take a snipe at homebrew, because I've spent more time than I care to admit sorting through the mess junior devs, designers, and non-technical people make of their dev machines with homebrew and when it comes time to upgrade everything inevitably goes horribly wrong.
The comment implies that real developers do not use Homebrew.
I've never knowingly used a tree structure since college, everything is abstracted away from me. Now whether this is a good thing or a bad thing remains to be seen, but it has not been a problem yet.
{foo : {bar : "baz"}}
The point being that hierarchical structures in programming languages abstract away from the underlying representation, and you can use them without knowing the details. (Though you should probably know anyway, so you understand your Big-O tradeoffs.)
The sizable problem with this statement is that it is using a solution to pigeonhole a person. It takes a bit of audacious hubris, something Torvalds (an extraordinary engineer no doubt) is known for, to paint an entire person and their capacity and skill into a singular and simple quote.
Here's a novel idea: sometimes a person is a good programmer and sometimes they're a bad programmer. There's definitely a skill level that people fall into, but these aren't neat lines delineated with tired generalisms about "worrying about code" versus worrying about "data structures". Put your question into an interview and get different people to ask it and sure pick the people who were able to get a better solution. Hopefully you're asking more than one question. That's totally OK, but please don't attribute someone's performance at one question or assume that you've mastered everything it takes to know to paint someone as a "bad programmer" from one interview loop[0] or performance with their performance for that one question.
Sometimes it's obvious someone has no idea what they're doing and you can tell that from a question, but even in that case they could become better. There might be a weakness in a fundamental area they need to learn and sure you don't want to hire them until they do, but that doesn't make them a "bad programmer". All it is is a specific problem that the candidate should recognize and improve on to up their skill. They shouldn't walk away from your massive ego rethinking their career as you puff your chess out and write a haughty blog post about what you think about people and their skills.
[0] http://steve-yegge.blogspot.com/2006/03/truth-about-intervie...
And yet I've never professionally had to reverse one or know if they are symmetrical.
That's what this question is actually testing: can you decompose a problem statement, expressed in plain English, into subproblems and then solve the subproblems?
Also, way to back up your claim that Homebrew is bad software by pointing to another guy not explaining anything about Homebrew but just dogpiling on Howell again.
Personally, I'd rather hire employees who are resourceful, can admit when they don't know something, can follow up with others and conduct proper research to tackle cases where they don't know the solution to a problem, and most of all, are a pleasure to work with and bring good and diligent attitudes about work into the office.
Hiring based on questions like this will get you someone ahead of the curve who perhaps you can trust to execute well immediately but will probably turn out to be a massive pain in the rear in the long run.
I'd rather hire people with good attitudes and willingness to learn and spend the extra time training them and helping them out rather than hire someone because they happened to get in enough rote technical exercises before coming into the interview.
https://medium.com/@kwindla/hiring-software-engineers-98498c...
(This is also on HN/new right now.)
It did however make me think of an almost opposite activity for the next time I interview someone:
Pick a difficult trick question (like Binary Tree reversal but harder/arcane) that neither of us can solve, and spend some time working on the problem together with the candidate.
After all, solving problems together (directly, or indirectly) is what we'll be doing day to day.
This might not be a good idea, but I'll definitely try it out once and find out.
Doing it on a computer is more along the lines of a software engineering problem, though still it is largely academic and counter to one of the tenets of good software engineering (that all code is a liability, and thus new/non battle tested code should be avoided whenever possible).
Dredging this comment, because while this article is not about whiteboard coding, it would inevitably be mentioned.
I think many people, including many interviewers, completely miss the point of whiteboard coding. I think this article needs to be shared: http://darrenkopp.com/posts/2013/04/19/Post-mortem-of-my-fai...
The key point is that the interviewer is, or should be, a friend. He should not be out to stump a person or out to get him; he should genuinely want a person to succeed at the interview. If not, it's better just to reject the person outright. White board coding should be seen as a way for person to engage in collaborative problem solving, which is something that we do on a daily basis. Can you explain your thought process? Can you admit your stuck? Can you have an intelligent discussion about your solution, listen to feedback, etc. The problem should be challenging and relevant, but it should not be a test of coding skill.
The other thing is that knowing a tree is symmetric and know how to reverse one don't count as new and unknown challenges that author mentions. Both are well documented and easily accessible to anyone who needs to know that information.
I recently had an interview where I was presented with a problem the company had to solve. That was new and exciting, not some minor detail from CS100 I learned years ago.
function reverse(t) {
var tmp = t.left;
t.left = reverse(t.right);
t.right = reverse(tmp);
return t;
}Asking candidates about algorithmic problems is great, if that's what you're hiring for. However, what has happened over the past decade is that a lot of companies who aren't even close to solving those kind of problems (e.g. digital ad agencies, web dev & mobile app shops) have started to ask these kind of puzzle questions in their interviews because "that's what Google does" and if Google does it, well, then it must be good.
Interviewing is hard and hiring the right candidate is even harder. A lot of interviewers don't even come close to having the experience of hiring a fellow colleague. A lot of these interviewers are basically hiring themselves meaning; if the interviewer has a PhD then the candidate they're looking should also have a PhD. If the interviewer loves Haskell then the candidate should also have this quality, if the interviewer can reverse a binary tree then, well.. regardless whether or not this has any application to the position they are hiring for,
When I interview I try to look for smart people who can explain and articulate problems well. I'm looking for people who are thoughtful and have a strong sense of integrity and an absence of ego. If they can talk about building software and programming I can easily spot if they bullshit or not. Smart people can easily learn how to reserve a binary tree.
If your software team works on code with lots of binary trees, interview questions about them makes a ton of sense.
If your software team is making a website that just displays stuff from databases and web services, interview questions about binary trees makes no sense at all.
Not really, because anyone reasonably competent could be up to speed on tree operations in an afternoon on the job. Interviewing them to see if they already know likely takes more energy than having them learn about them.
That holds true for lots of things. "You say you haven't used GitLab. Too bad: that's what we use so we'll have to hire someone else." "Huh? I'm pretty sure I can figure it out in a couple of hours." "Nope. We need someone ready to hit the ground running."
The truth is that interviews should test the skills your job actually requires. For a variety of skills outside of that core, it is valuable to have someone on your team who understands it for other people to go to. But if you're not doing algorithms all the time, you don't need or want everyone to have mastered that.
And I say this as someone who knows algorithms well enough to pass his interview.
Except you've done exactly this by selecting tree structures, something you find intuitive, and using it as a measurement for everything else. If you hire like this you will be successful in surrounding yourself with others who think just like you. Maybe that's a strength for the product you're building but it's a strength achieved by limiting yourself to a narrow set of knowledge.
Its never that simple, there's a lot more to consider, and the vast majority of candidates will bomb a technical question or two eventually.
For those of you that are job hunting right now... don't get bent out of shape, there are plenty of employers that won't hinge their acceptance on one technical brain-teaser.
Also, the first definition of symmetric tree is wrong. One more appropriate would say that the left branch is the mirror image of the right branch.
His herpes analogy is ridiculous. And I've used brew since getting a Mac and no I'm not suffering now.
1. If your company does it, you should try to end it. Research alternatives and propose them. Take-home problems that mimic real problems at your company might be an option, as would hiring someone for a few contracting hours to do solve real problems before hiring them full-time.
2. If you apply to a company that uses these questions, politely excuse yourself from the process (assuming you are at the point in your life where you can refuse offers). That company probably doesn't really have the employee-focused culture you'd want to be a part of. If enough candidates refuse to do these tests, we might see them decline in prevalence.
3. If your company used interview methods like this and then stopped, try to create a case study. What did your company switch to? Do you have data to back up the perceived benefits of the switch?
That does not mean the candidate will be able to design clean APIs, know how best work with different types of data, or even how to troubleshoot existing code.
Still, pat yourself on the back, I mean, your entire team can answer a specific trick interview question, and that's all that matters, right? Right?
Sigh.
In the first approach he gives, you need to divide the problem into smaller subproblems, knowing that a tree is the combination of either two branches that are themselves trees or a single leaf node. What's the rule for determining whether two trees are symmetric? Well, if both of them are leaves, the answer is trivially true. If one of them is a leaf and the other is a branch, the answer is trivially false. If they are both branches, then you need to make sure that the left branch matches the reversed copy of the right branch, and vice versa, i.e. left is symmetric to right and right is symmetric to left. And you have the solution he gave.
That also clues you in to the second solution, where you define "symmetric" as "the reverse of tree1 is equal to tree2", and then write appropriate recursive helper functions for this.
What the author's really testing for is "Can you decompose problems that you don't know how to do into problems that you do know how to do?" This is a critical skill for doing anything new, and the vast majority of economic gains in the software industry go to people producing new stuff and not looking up other peoples' solutions. Effectively, what he's trying to select for is folks who don't believe this is a matter of received wisdom and instead are willing to figure it out themselves.
(Interestingly, there's a bug in his code. It doesn't affect the correctness of the point he's trying to make, but would be infuriating to an actual Javascript programmer working with this function. See if you can spot it before he merges my pull request...)
Sure, you don't absolutely need to know the problem to be able to derive an answer on the spot, but the problem with a widely known question like this is that candidates who already know it will have an almost insurmountable advantage over those who have never heard of it before, which makes it in essence a "know it or not" question.
There is very little you can do to distinguish between the candidates who knew the question beforehand but feigned ignorance versus those who actually managed to figure it out on the spot.
The article author doesn't think of his question as a "know it or not" question, but a "you should be able to figure this out" question:
> Here's the first reason why this question is so good: a skilled programmer can map their intuition to code. You can tell at a glance whether a binary tree is symmetric, but beginners will often struggle with a question like this because they approach this problem from a visual/intuitive angle rather than the logical angle.
I think it's a fair criticism to say that, at this point, "reverse a binary tree" is something a lot of applicants will have simply memorized rather than figuring out on the fly. But (to play Devil's Advocate) isn't the goal of the question - to test a candidate's ability to think logically and problem-solve on the fly - a reasonable one?
It seems to me the real problems with whiteboard-coding interviews aren't that what they're trying to test - problem-solving on the fly - isn't useful or valuable, but that whiteboard coding is confounded by so many irrelevant variables: ability to think under pressure, ability to code "in your head" (without an editor, IDE or REPL), memorizing solutions, and so forth.
Programmers are so sensitive, and honestly all your "sighing" is obnoxious. You don't have it all figured out.
These kind of tests are aiming to find developers who can think out of the box and start solving problems they haven't heard of before. This is quite a simple problem to solve if you start thinking about it. If you get stuck because of the binary tree and because you've never done that before, then I believe you would be a mediocre dev (which is not a problem in many companies).
It's an excellent heuristic. It doesn't separate good vs bad programmers; it identifies programmers who love solving problems, and see their careers as problem solvers instead of as limited-view coders who are assigned a javascript task.
Like you said, it also takes time to develop this. So it also identifies programmers who have taken the time to hone that kind of problem-solving intuition, which is far more difficult to develop, than throwing up a webpage with bootstrap.
I run a successful coding interview prep bootcamp for a living. Among other things, we also go thru several Data Structures and Algorithms. Primary objective is to practice intuition on some of these problems. Those who work hard at it, invariably develop irreversible intuition to this stuff.
Perhaps instead of being annoyed, you could try to understand why people are sighing. Many of them will have lost out on a job they were qualified to do because of bullshit interviewing practices like this one. It has a real effect on them. It's not a trivial issue.
> You don't have it all figured out.
No one here has claimed to have it all figured out, nor are the majority of comments an implication of that. We're all just saying we have one thing figured out: whiteboard interviews are terrible, alienating, unrealistic, antiquated tools for hiring.
Your profile says you're a CTO, and you should know that research increasingly shows that people don't quit jobs, they quit bosses. If you want to be the kind of boss that programmers want to work for, you should be more open to understanding why the vast majority of us feel strongly about a topic like this instead of calling us obnoxious.
If you yourself are not a programmer, I'm not sure why you're weighing in on this at all.
a. dodged a bullet b. should basically do the exact opposite of every part of their hiring practices