There are many variations in questions too: search a tree or graph with some customized criteria, using a double buffer to implement a tree-style prefix scan, implementing an integer multiplication with unlimited digits, some streaming algorithm, tree-walking to transform one tree to another, a simplified skip list, the options are unlimited. A good candidate tends to grasp the high-level concepts quickly (and they can ask questions), and is quick to convert intuition into working code. I find that there is a strong positive correlation between the performance in work and the performance in such coding interviews.
To me, the value of a software engineer is to ask questions, make hypotheses and be able to iterate quickly. Balancing trees, leetcode and other algorithmic stuff on the spot sounds like bringing the dreadful education system structure to the real world.
Also if a senior person can't in 30/45 min of talking with someone figure out the general experience level then the problem is them, really.
This is why I've always been so confused. Why is the software engineering interview wildly different from the traditional engineering interview (seniors sit down with candidates and discuss how to solve a relevant proxy problem the team is currently undergoing (has the side benefit of making the interview process potentially fruitful if you don't go with that candidate. This can be (and sometimes is) abused though)). I mean... we all speak the same language, and that isn't standard English... right?
The thesis is not that these exercises are representative of work but rather predictive of performance.
Sales has a similar phenomenon with sports. While there is no athleticism involved in selling, many believe history in competitive sports to be a positive predictor of sales success.
---
You can reasonably argue whether leetcode accomplishes this well or poorly, but...
Always remember that purpose of an interview (for an employer) is to predict performance. So you are looking for the resume screening+interview process that most accurately assesses that.
OK take it as read that your posturing has succeeded and we all agree that you're a brilliant interpersonal genius and the rest of us are all useless chumps. What then? The rest of us still need to interview and make hiring recommendations. Or are you suggesting that employers should fire anyone who lacks your magical talent?
- Future proof. Unless I work for an outsourcing company, sooner or later I will want to push the envelope, or so I hope I do. And to push the envelop, one needs good CS fundamentals (maybe there are some exceptions in some specialized field). Think about React. It's a JS framework, yet to invent it one needs to understand at least compiler and graph.
- Geekiness/talent filter. The same reason that the nascent Google and Microsoft and any elite companies like Jane Street asked Putnam questions, Martin Gardner questions, ICPC questions, and clever probability puzzles. Whether it's a good idea is debatable, but at least those companies want to have such type of people and they were hugely successful. Note the word filter: people who pass the interview may not be good, but failing the interview means the candidate may not be smart or geeky enough. Again, I'm not endorsing the idea but just exploring the motivation of the interview policies.
- Information density. Assuming a company does want to time box a coding interview in an hour, it will be hard to come up with realistic question without spending too much time on the context. On the other hand, it's relatively easy to come up with a data structure/algorithm question that packs enough number of abstractions and inspection points to examine one's programming skill.
Actually most of them, including the really inexperienced juniors have 'figured' you out in less that 15 minutes, or at least they have decided whether to hire you or not in 15 minutes. But they have to put on a charade of being fair.
Also a 'white' older male is the least preferred even if he is smarter compared to all females and the minorities that are being interviewed as long as they are not terrible. Biases galore.
Or it's not their call, theirs is just to work with them after.
I've had to work on tree traversal stuff multiple times in my life, anything low level GUI related will work with trees a ton.
I've also had to work with hash tables directly, and with memory caching layers.
I really should learn to write a proper parser, as I've had to write parsers multiple times now and they are always an ugly hack job.
At some of my past jobs and the current one, this kind of algorithmic knowledge was important to build features that were differentiators in the market. As much as people love to pretend, not every single possible solution is in a library. Sometimes you're the one building the library.
It doesn't have to be leetcode, but candidates should at least be able to produce some code that doesn't come from the README of their favourite framework.
Also, talking for 30/45 mins can be enough, but it produces false positives when you have people coaching candidates. I've had people completely acing interviews that it felt like the perfect candidate. Well, it was rehearsed. When I asked for a fizz-buzz type question they completely messed it up.
Lots of people have lots of ideas on what makes a great interview question, but none of them are backed by data. LeetCode-style algo questions USED to be an indicator of intelligence as per Google's investigation, but now it's so heavily gamed that I doubt there's any signal behind it anymore.
Someone needs to spend time and effort and money and actually do random controlled tests to see if people who pass a particular type of test actually make good employees. But so far no one has done this, except Google as far as I've seen. But even now, I think there's no evidence that even algorithm questions are any indicator, given what I've seen.
This random controlled trial you're talking about is so hard to do because there are infinite confounders. How do you even measure success? After how long? How do you separate politics from technical ability?
In reality people above a certain bar are able to make some contribution to most projects. You can't and don't need to staff every project with superstars. The superstars are rare but are not hard to identify. They'll have the track record, reputation and the skills.
Another thing to consider is that the best outcome of interviews depends on the quality of the pipeline into those interviews. If you can't get good employees to apply you won't get good employees hired. Pay well and build a company that people want to work for, and then your hiring is going to be easier.
Do you have evidence that the standard coding interview works? (There's evidence that it doesn't)
I'm with you that the claim might be too strong to say "this is the way" but that's because I'm of the (very strong) opinion that interviewing is an extremely fuzzy process and there are no clear cut metrics to measure one's abilities in a significantly meaningful way. a̶l̶l̶ ̶m̶o̶d̶e̶l̶s̶ ̶a̶r̶e̶ ̶w̶r̶o̶n̶g̶ ̶b̶u̶t̶ ̶s̶o̶m̶e̶ ̶a̶r̶e̶ ̶u̶s̶e̶f̶u̶l̶ There are useful interviewing methods but certainly not "the best" method. Trying to just mark checkboxes only leads to mediocre results. The reason we're generally okay with this is because we more often than not don't need rockstars and it doesn't make sense to put a lot of time and energy into this process when we get sufficient results through lazy methods.
FWIW, a typical (non-software) engineering job really just involves high level discussions like the OP suggests but even without the implementation. It is generally about seeing how the person thinks/problem solves and looking at problems they solved in the past. It isn't resource intensive and good enough. Because truth is, you don't know what someone is like as an employee until they are an employee (and even then this is fuzzy)
Where can I find the data on this?
I want to see if they can think in code. If they can translate something simple to written code. There's always a chance to have a conversation as well to probe different aspects of their understanding.
A few of those by a few people and I think you get a pretty good read.
Mix in some questions about their work and/or some general knowledge. I've also given people code to read (real world code) and explain to me.
For Algorithms and Data Structures I can look at their grades if they're fresh out of school. But we all know most of our work is not that (someone's gonna show up on the thread and say differently - I'm sure ;) ). If you can think in code and have the right mental models implementing an algorithm from a book isn't hard if you really have to.
But it won't cut it for a senior. Such people should be able to answer a wide range of questions, including some algorithm and data structure stuff. Not that they have to be able to code a min-max-heap from scratch, but they have to know what's out there, how to use it, and how to keep the overview of an entire project. That's not going to be evident from a coding interview.
So, horses for courses.
A typical engineer doesn't begin their day thinking "I should implement a streaming algorithm" out of nowhere (and if they do, they can always seek reference). They analyze a (typically underspecified) problem, figure out the constraints, and then start thinking about a solution. Note that both processes, analysis and solution, can take hours, days, weeks or months!
Coding interviews have everything backwards by providing synthetic solutions to mostly out-of-context problems.
For example, interviewers could:
- Share some portion of a codebase with the candidate prior to the interview
- Go over the code with the candidate
That's it. You will know really fast who's senior and who's never wrote software. It can't be faked. And it can be adapted to mostly any positions (backend, frontend, devops, architecture, whatever).
I've hired and interviewed a lot of people over the years. Mostly without the help of recruiters. I like to think I'm pretty good at that, actually. I always look for people that are eager to learn. I care more about what they don't know than what they can regurgitate because they did lots of silly online prep. If people are young and fresh out of school, my assumption is they are going to have to learn a lot in a hurry. So I look for people that are curious, eager, and have an open mind. The best way to do that is to get them slightly out of their comfort zone and talking about how they would tackle things. The signs I look for here are enthusiasm, ability to surprise with good answers, etc.
This generally does not involve code interviews; though I might ask some targeted questions about how they would deal with certain things in languages, frameworks, etc. they list as being proficient in. I've of course worked for some people that insisted on me organizing code interviews and it's a chore and IMHO you learn absolutely nothing that isn't obvious if you just casually talk to people for 30 minutes or so. I usually argue against such things and prefer just talking to people directly.
Thus happened to us twice, and after we introduced a simple live coding session to our interviews where you have to implement a simple real world web component (for a frontend position) suddenly the problem of bad hires disappeared almost entirely (you can’t judge someone’s character during a short interview but that’s another issue).
I have no idea what these people expect. We ask coding questions, we have basic programming skills as a requirement, what do they think will happen if they get the job? Do they expect to just magically learn the required skills or hope that nobody will notice that they're not able to close their tasks?
Although i see it as a way for the candidate and i to talk about code in general, and assess his level. No as a simple barrier.
i hired a candidate that completely failed a code interview because he was super nervous, but just talking about the problem made me quite sure he was actually good.
It's surprising how many 'senior' engineers are actually BSers who will slow you down or derail you completely rather than speed you up and you need to spot them because they will excel at getting through the non-technical interview filtering!
Also, I'm interested in how you explore things and explain things. I'm not actually interested in acquiring an implementation of FizzBuzz or whatever. I just want you to show me that you 'get' it and then we can get on to the interesting stuff like 'tell me about your last project' etc.
So don't be too hasty to think the people doing technical interviews are idiots thinking devs are interchangeable cogs etc.
Sounds like you'd more valuably/realistically get that from the discussion of a previous project though? How it works, or something interesting they had to figure out, etc.?
> don't be too hasty to think the people doing technical interviews are idiots
I don't think anyone has a problem with technical interviews? At least I agree that's not reasonable. That doesn't have to mean 'coding' though, you can ask about how they'd approach a particular problem, quiz on some fundamental knowledge in a fizzbuzz sort of way, etc.
You can more easily tune it to the kind of candidate you're looking for then too, for example I've asked how they'd tackle improving the performance of a particular SQL query that's been identified as too slow. There's a tonne of possible answers to that ranging from naïve/they don't really know, through pragmatic good fit responses, to way overkill I hope they understand we don't need them doing that here/not operating at that scale etc. - and it's fairly open-ended in what you can discuss driven by what they volunteer and know about. (Which is another good thing IMO, I don't like being on either side of interviewer quizzing and candidate umming and ahhing not really knowing! Both more comfortable and more beneficial to have a discussion about whatever is known IMO; to quickly move along the perimeter if you hit the edge of that.)
There are many people who spend more effort creating the illusion of competence on paper and on the job, getting harder to detect the higher they go.
We as a profession (software engineers) have continually resisted broad unified certification like other engineers which could be a replacement for code interviews to assess competence, but would have other drawbacks.
So we are stuck with code interviews to ferret out BSers. And even then it sometimes fails. But it appears to be the best tool we have, because there is nowhere to hide. Don't take ot so personally.
If you can navigate a software engineering position, purely undetected, by bullshitting it, I would say you can be a very good manager. You can probably handle high level concepts without knowing the implementation details.
Or maybe their strategy still catches all of the frauds and it has therefore never been a problem to them?
I have to agree with their take, and just asking a bunch of technical questions -even without any code- is good enough to filter out the obvious incompetents.
How can a code interview fail? Hidden earpiece?
Your approach is a major outlier. You probably have many good candidates turning away, and for good reason, they have no idea that you are different to everyone else. Find a different way to do this, there are several other approaches.
It's almost like everyone big and small is standardizing on this model. Feels like one of those mandatory courses you took in college: the teacher and all the students knew it was bullshit yet you needed to perform the parroting at some adequate level to pass.
I have 21 years of varied experience in software engineering, yet a recent "technical" phone interview was a kid asking me to "balance a B-tree"; I can tell his expectation was for me to start reciting some CS algorithm BS, probably that's what everyone else is doing. I politely declined and that was the end of the interviewing process with that company.
The conversation is the most important part of the interview, and the thinking (and communication) is the most important thing I'm trying to judge after basic skills.
Like you said, you can get a good sense within the first few lines of pseudocode if someone's at least competent at writing code. But that's just one motivation behind coding questions.
It's also very difficult to talk about code, algorithms, and solving problems without a concrete problem and concrete code in front of the candidate and interviewer. So both the question and the code the candidate writes are mainly context for the conversation where I try to see how the candidate thinks.
These kind of articles make me sad because I (and many other interviewers I've worked with) try to make it clear that this isn't a test - we don't care so much about being "right" or "wrong", and there shouldn't be any tricks of "a ha" moments.
We explain the goals and what we're looking for right up front. And I would hope most interviewers do the same, but I guess not. So there's this persistent myth among both interviewers and candidates that coding questions are about getting a right answer.
That's a shame because coding questions get such a bad rap, but I'm not really aware of a better options. Take-home problems and looking at GitHub are unfair to many people. A well-run technical interview should give lots of people a chance to succeed.
If the interviewer said at the beginning, "I don't expect you to solve this problem in the 40 minute nor to have an optimal solution. I just want to watch you write some code and hear the problems you foresee and how you'd solve them" then maybe I could relax and do that. But, generally the pressure is on "get this right in 40 minutes or you're rejected"
After writing this article it became very apparent to me that I'm complete garbage at interviews, but I'll outperform and exceed at the actual job function.
> These kind of articles make me sad because I (and many other interviewers I've worked with) try to make it clear that this isn't a test - we don't care so much about being "right" or "wrong", and there shouldn't be any tricks of "a ha" moments. > We explain the goals and what we're looking for right up front. And I would hope most interviewers do the same, but I guess not. So there's this persistent myth among both interviewers and candidates that coding questions are about getting a right answer.
I understand all of those things. I've written the same before[1]. However, as clear as your instructions are and as well meaning you may be, it may not help. I can logically understand every word you say, but as soon as that question rolls out, I will now be dealing with stress hormones and 30 years of learned behaviors from thousands of experiences, whether I choose to or not.
So while I applaud your methodology and wholeheartedly agree, just telling people that doesn't guarantee that it's not still an issue because humans are complex organisms.
[1]: https://darrenkopp.com/posts/2016/02/25/always-learn-somethi...
We would give them a whiteboard problem, but:
a) it was a simple, stupid problem in C (C++ was our implementation language, so thinking at byte level was an important skill)
b) we were very generous about minor mistakes like missing semicolons, etc.
c) we were very generous about "points for effort"; if they didn't make it through the problem but we saw that they were on the right track we might pass them. Total frauds outed themselves very early; they would produce between jack and squat in terms of actual code (a lot of bloviation though).
But again, most companies aren't that company, or your company. For most screening coding exercises, a correct answer (and even something like optimal algorithm complexity) is a must to pass the candidate.
Then, in the post interview roundup we talk about the candidates and you’re a bit disheartened that they didn’t complete the exercise, so you give a pass even when every other person in the room gives a thumbs up.
Nah. Re evaluate yourself and your biases.
The issues people have with coding interviews are more about the interviewers than the questions, honestly.
What about the lad who develops homebrew who got rejected from Google because he wasn't able to invert a binary tree on the spot? Many Googlers use his software internally and externally. If the purpose of the interview is to make sure he can code why did he fail?
https://www.quora.com/Whats-the-logic-behind-Google-rejectin...
He seems to have a great attitude and would fit right in but it's clear Google is optimising to keep people out rather than find great software Devs.
I also dont see how this one anecdote (even if it was true) invalidates anything I said above. You’re gonna have false negatives in any system you chose unless you just wave everyone in
From all his public complaining about failing an interview it seems Google did the right thing not hiring him, he has a massive ego and it's very possible that "writing homebrew" is less useful to google than "inverting a binary tree"
He said 90% of Google engineers used Homebrew. Google engineers said it wasn't true.[1]
He said Homebrew using OS libraries saved a lot of pain. He presented it as an example of why Google should have hired him. Actually it caused enough pain Homebrew stopped it.
Way too often, this results in a painful slog, lots of hints from me, and, nowadays, snippets of AI generated code copied in that actually solve part of the problem but that they don't read and therefore end up breaking.
I believe there is no single declaration about 'what the interviewer' wants.
Especially that several times they do not know themselves, just try to mimic what is expected from them or seen in some random situation they came across. Sometimes, of course.
---
"brining key stakeholders into conversations at appropriate time"
This is some very good quality euphemism here, two thumbs up! : ))
And IME this is what most interviewers have been taught. They've got a list of sample questions, they've been told that if they give the knapsack problem and the interviewee doesn't immediately call out 'dynamic programming' than the interviewee is a pass.
If you only want to see working code than you are the exception rather than the rule.
I don't expect you to quickly crank out a novel optimal algorithm. But I like to see that you can think about how to solve problems programmatically. I would like to see you can identify differences in different algorithms and what tradeoffs there are. Considering different approaches that may be faster also shows that you didn't just memorize an algorithm from somewhere but that too took time to actual understand the problem and build a mental model in your head.
I have given people great reviews when they couldn't come up with a working algorthim because the clearly knew how to think like a programmer and considered all of the important things.
They only want to see somebody who can get working code and a glimpse of their thought process. But from 100s of mediocre examples, the better coder will have a "better thought process."
Same goes for dating. Of course people will swear up and down they "only consider personality." Turns out, they've met 10 other people with a better personality than you.
Just because they're "only looking for x" doesn't mean they'll accept anybody that clears the bar.
The ultimate read between the lines though is that "oh I'm only looking for xyz, nothing superhuman" in a process where you have 10,000 competitors and applicants will still require high performance on your part. It's just a nicety, a meaningless phrase.
Has everyone I hired got "perfect marks" on the test? Of course not! It's not about that. It's about seeing how they react to a problem, watching them break it down, ask questions, and, ultimately, get on with it. If the job is too sweep floors you need to be able to hold a broom. It's as simple as that.
I don't have this problem - I can easily tell who is good and who is not good by looking at their stuff online, which repos they are contributing to and what their contribution is. I look at personal projects - I can easily tell what parts they wrote vs didn't because it's usually specific to the project.
I can tell from their blog posts and comments, especially GitHub comments - I can even see if they're pushing features at 11PM on a Friday if the obsession piece is crucial to the hire.
People who say all their work is hidden under NDA or they're new grads and haven't done anything yet - sorry, if there's nothing to view online I just wouldn't qualify you to interview.
Though I have given out 50+ code tests in my career because I had to, I would never choose to do this, if given the chance when hiring someone I would never give them a code test. I think it's an amateur move and wastes everyone's time. At its best (as in the case of CTCI interviews) it's an exclusivity filter for academics who memorized the optimal data structures for algorithms as taught in school, but the candidate might not have any of the skills needed to build app features, perform DevOps, etc. or even operate a Terminal - CTCI doesn't cover anything async, nothing about UIs, APIs, databases, services, git, design, file formats, etc. it's purely academic sport. And like I said, a good developer's work should be highly visible anyway - skip the random code test.
I would spend the recruiting effort finding specific developers using specific technologies that aligns with the role and making them excited about the opportunity rather than canvasing 1000 code tests out to anyone who applies.
Lots of people skate by in technical roles by barely doing anything technical. Lots of people also overinflate their achievements in resumes and conversations aka lying. How is non technical recruiter supposed to evaluate their coding chops?
> People who say all their work is hidden under NDA or they're new grads and haven't done anything yet - sorry, if there's nothing to view online I just wouldn't qualify you to interview.
Lol, great method! I have a better one - just hire acm winners. No need to test them
The goal of an interview is to find out what the candidate can do.
If you interview someone, and they "fail all the questions we asked", it is not the candidate who has failed... the interviewer has failed. You ran an entire interview and all you know is what the candidate can't do. You have learned virtually nothing. The questions must be adjusted to what the candidate can do. Only for the absolute worst candidates should you ever exit an interview saying they failed at everything. (Sadly, such candidates do exist. For instance, consider the recent stories about AI fraud. If your level of programming skill is "I have ChatGPT", I'm going to have a hard time scoping a question down to you.)
If I had asked this question in an interview, or a related one, I would have stepped the problem down until the candidate could pick it up. (Odds are I'd have started lower and worked my way up anyhow.) If the candidate then sort of finds their footing and once going can start folding in the other requirements as we go, great. Who sits down and designs a system for all twelve adjectives ("reliable", "concurrent", "redundant", "runs on a Z80") we want anyhow? One can not help but design a system one adjective at a time, with the others only kept in mind to make sure we don't back into a corner. There's no reason not to interview that way too. (And I tell the candidate this is what we're going to do, so hopefully they don't feel like I'm out to get them or experience requirements whiplash without knowing why I'm doing this.)
Is that important to you? Maybe it is for some people. As a person who has been in team lead and hiring capacity before, it is not for me.
I trained for interviewing at Google twice, but never really chose to give interviews despite that being a highly pressured part of the job because I could not philosophically vibe with that process. But what some people who have adopted this process are missing is that Google etc does this only because they are swamped with resumes and have their pick of bazillions of quality engineers. They don't care about false negatives, more false positives.
Startups and smaller companies should care about false negatives. It's hard to find and retain good people. Smaller companies need to aggressively find and cultivate good people to make good teams in order to be/stay competitive -- and that means accepting a diversity of ways of working.
I've worked on a Python interpreter (not CPython) for many years now, I just never used that particular module (because we do it differently in our implementation). I suppose businesses try their best, but aren't always measuring what they think they are...
Wait, is Z80 seriously something that you want/design for?
https://tvtropes.org/pmwiki/pmwiki.php/Main/ArsonMurderAndJa...
1. My preparation for an interview involves researching the company, not technical matters. I don't brush up on coding interview questions. I've never done leetcode.
2. If I find the interview questions to be ridiculously off-topic (such as silly algorithm questions), I end the interview. You're not the kind of company I want to work with.
3. If I find the questions to be valid, but I can't answer them, then I'm not the right candidate for the job (hopefully I'd already have found this out during the research phase, but we all make mistakes).
4. If we can get past all this gatekeeping to the actual important topic if what BUSINESS issues they're trying to solve, and how I can fit into this process, then we've got a real interview and I'm interested.
So far I haven't been out of work more than a few months.
If you can't determine from a few interviews you want to hire me or not, it's a no from me. It's a HUGE red flag if an employer can't seem to make up their mind so they have to resort to awkward "test drives" like this. If you have a job at the time already, it also makes it almost impossible to pull off without using PTO.
Add all these things together and you really don't want to work for a company who is very inexperienced in hiring such that they resort to tests like these. It shows they don't know how to interview, are not efficient with your time (and probably their time also), and a high chance they are non-technical.
There's a bootstrap issue as well. A developer with a brand name on their resume is much more desirable than one without, even for equal skill and experience.
I generally ask the purpose of the question.
Sometimes there are valid reasons for why they look at very strong algorithm skills and then I simply admit it's not really my cup of tea nor my passion.
Sometimes they answer variations of "it's standard/it's how we do it" so then I propose whether we would like to code something more similar to what the daily job would be to which they generally play along, even happily.
But if some don't I then say that I don't see a fit.
Might be awkward but...who cares? I don't, they won't after 5 seconds the interview's over either.
- people with licenses still do interviews, often just as grueling
- licensed careers with high performers (lawyers, doctors, ib, etc) have other forms of filtering which are much more painful, like years of low pay internships
Studying for a few weeks to solve fun puzzles to make 400k sounds like a deal to me.
The number of people who "aced the interview" but are borderline worthless for real product development is not trivial in my experience.
Agreed, I never asserted against these two points. The point of licensure is to make the first round, which is fairly routine at this point, more equitable and less susceptible to probabilistic effects. It also frees labor from administering this exam round to every candidate.
The licensed engineers I know think software interviews are insane.
Real licenses do not expire without notice. Real license exams are more consistent, have clear pass criteria, and have higher pass rates.
It isn't really a licensure examination because, as you point out, the industry doesn't bother to put the resources in to make sure anything is systemically analysed. It is a process to balance supply and demand of jobs.
Those are your top two problems? When was the last time you saw an exam where the proctor was expected to hold a conversation with the examinees?
Given that Medical boards and the Bar strike fear into the hearts of students and demand immense preparation, I think we could do pretty well. The test prep mentality isn’t altogether different from the “grinding leetcode” that happens today anyway. The difference is that it would at least be fair.
There’s the easy exam that pretty much everyone passes eh e they get their degree (the FE), and then there’s the hard one that not even everyone attempts after a couple years of experience (the PE).
And within each level, you specify your discipline (civil, mechanical, etc) and then are required to have deeper knowledge of several subfields within that discipline.
It would also be easier for the industry to settle on some filters in the exam to block people who just SPAM job postings. For example, open jobs for doctors are only posted on websites that are available to doctors. There is no way for the general public to SPAM job listings for doctors.
Candidates who can code well can point out code that has obvious problems. Just ask if this is good or bad, and if it is bad, how they could improve it. This demonstrates competency and doesn't make the interview seem like a grind but instead more like a conversation.
The best code review interviews are the ones where there is a healthy amount of actual code, with a handful of functions and classes, some badly named variables, bad comments, some misleading code paths, couple bad patterns, etc… the worst ones are a non-optimal solution and you’re asked to make it optimal. That’s just leetcode disguised as “code review”.
"The martini may be simple, but it is not easy to make an excellent one. It's a very solid test of a bartender's skill because, unlike many drinks, ingredients alone cannot carry the cocktail. A piña colada for example, is mostly about ingredients (are you using a good coconut cream? fresh pineapple?) For the martini the chilling and dilution need to be just right. This tests the bartender's most important skill: mixing. Proper mixing of the beverage is ultimately what makes a martini."
[..]
"martinis are shockingly easy to fuck up. and this conversation is exactly the reason why the martini is a good test of a bartender's capability. being a bartender is more than putting fixed quantities of ingredients in a glass. how do you know when your martini is properly diluted, either by shaking or stirring? a good bartender will know. a bad bartender will not. a terrible bartender won't even realize dilution is crucial."
I don't really drink much and never had a martini in my life, but I thought it was pretty interesting.
I know not everyone works with CSV necessarily, but there are dozens of libraries for a lot of languages. Even if focused on N being those supported in a given company/org. It should be less than an hour of work. Bonus points for any tests/automation, etc.
And the patch:
- does fix something but not the described bug.
- could do the same thing in a third of the added line count.
- has typos or other errors.
Last time I got hit with an interview question like that, my answer ultimately had to be "block the merge and counsel the person who wrote this about performance." I'm still not sure if that's the answer they were looking for (this was for a staff engineer position), but I'd stand by it 100% of the time.
I once had a coding interview like this, but the problem was that the code was so obviously bad, I couldn't even make sense of what the code was supposed to do if it were good. It felt like the interviewer had just come up with an example of bad code without any context of how the code would make sense if made "good". It was just totally artifical.
If someone had presented the bad code in some Stack Overflow question, I would have started by stepping back to ask, "What are your trying to do?" Except in this case, the interviewer wasn't actually trying to do anything except quiz me.
Identifying a bug in production code would be better, I think.
This was an effective method to screen out applicants who didn't have the basic coding skills to align with their stated resume experience. And there were a decent number of these.
We did further development project exercises later in the process that took about 15 hours. We paid the candidates for this time, even those that didn't pass. It was also an effective screening tool.
All our exercises were very "real world." In the candidates own development environment and having been given instructions on how to prepare. They also have access to whatever Internet resources they want while doing the exercises. If they can't do the exercises, they can't do the job.
I know there are mixed opinions on this and I feel for candidates who have to invest a ton of time in exercises like this. But I can't imagine trying to hire without visibility into how they execute relatively basic software development tasks.
I think employers can and should structure the process so the time investment is minimal upfront and only increases as both parties have gotten to know each other and want to proceed.
Truth is, I'd much rather hire someone who is interested in working for us for 2-4 weeks as a contractor instead of the normal interview process. But that usually doesn't work for the candidate.
Absolutely. They do leet code -- we must as well. Google is laying people off -- so must we. Steve Jobs was an asshole, I also must be an asshole, that's how you grow a great company.
> The question, which I’ve heard is a Facebook favorite, was “convert a decimal number to base negative 2”.
Assuming I don't care as much about the the interview and am just practicing, I would have asked "so how often do you folks, convert numbers to base negative two?"
> but fuck that question and just waiting for you to eventually arrive at the little trick to make it work.
That sounds like a "coffin"-type problem as per https://arxiv.org/pdf/1110.1556. You have to know the trick, or you might spin your wheels for 40 minutes.
My worst interview questions that were silly:
• number theory for django dev: how many prime numbers are there.
• random brain teaser for django dev: infinite lasers pointed in space that turn at each other at the rotational speed of light- does the intersection travel faster than the speed of light
And questions that were very fair that I bombed:
• ms paint fill method on whiteboard. I think I checked diagonals when I wasnt supposed to - i cant remember but they were not happy with my answer.
• for a given phone number and a US dialpad what are all the possible letter combinations for the number? In python this is just the built in cartesian product. But I still struggle to write it from scratch with a recursive function and getting the accumulation correct.
I very much dislike the interview questions which roughly translate to "do you know combinatorics/<insert-some-other-area-of-math>?". Even back when I was personally good at those questions (straight out of University), they were less interesting interview questions.
I'm sure there's some jobs where it's relevant, but for the vast majority of software jobs... not so much. I get why it's done - they can be small, self-contained problems, which don't require a lot of external context. And they can be easier to come up with.
> The problem wasn't the question, though. It was that the person asking me this didn't even know what "yield" was in Python.
That is a problem too. As an interviewer, I'm okay with someone using a language or lang feature I don't know, so long as they can explain it. Some interviewers are much less flexible. Hell, sometimes I'll feign ignorance over some lang feature just to see what their explanation is like, because it's a measure of signal on how well they can teach a feature to a more junior dev.
In what real world situation would you even use a new base system?
Seti and Aliens are the only one that comes to mind, encode a new numerical system based on some fundamental physical constant like the atomic weight of hydrogen.
> Seti and Aliens are the only one that comes to mind, encode a new numerical system based on some fundamental physical constant like the atomic weight of hydrogen.
That's base-1, aka counting
The trouble is it’s a hard problem, and it usually gives -some signal, so it’s sort of better than nothing? I guess? In cases where contract-to-hire make sense for both the company and candidate I generally regard that as ideal, but that’s not every situation.
Someone will solve this, and that person will be very well-loved.
There's really no substitute for interviewing based on candidate's experience.
Apparently the new job market trend is applicants firing off dozens/hundreds of AI generated applications, which are then screened by an AI on the other end.
With that said you made me laugh out loud because I have a friend who knew two people trying to LLM some minor contract and it just went on forever. I definitely think two LLMs talking to each other with two people copy pasting and not realizing it is worth like, a Rick and Morty episode or something.
I interviewed at Google and all the questions were practical, challenging, and not found on leetcode.
I've seen all kinds of interviews in my twenty years of experience, and while all types can be done poorly, (including DS&A ones), having some live coding is one of the best signals you can get in a short amount of time.
I've been blown away by a candidate while they're talking about their experience, but then asking them to code something small, they utterly bomb.
Conversely, I've seen shy, humble candidates struggle to articulate their skills, and then crush harder and harder coding challenges.
That always happens in an interview loop too though.
I have no insider knowledge, but I always wondered if those kinds of cultures had a hazing / blind-leading-the-blind aspect to them. I.e. the people who got hired were the ones who jumped through some arbitrary hoops, so they doubled down on those hoops being the right hoops to choose the best candidates.
What was once considered a test of "intelligence" and skill in coding (whether a true assumption or not) has just become a test of "how much is a person willing to struggle to earn entry into our hallowed halls". Actual potential ability in the role is secondary to getting the role.
At least part of the problem is that leetcode questions are easy to ask, and most interviewers don't want to go through the hassle of coming up a question that scales well to the candidate's experience and knowledge.
The same screen can be done with much simpler problems. This light coding interview should also be more or less pass/fail. Do it before everything else to short circuit those who have no idea how to write a for loop.
Save the interesting insights for real problem solving, code reviews, systems design… DSA under pressure is not something that actually happens on the job IME.
Still, I’m conflicted, because a solid DSA understanding is incredibly helpful at times. At the very least, a solid understanding of the commonly used data structures, how to transform data structures, a good understanding of memory allocation with respect to code and runtime, reference vs copy… All things that if not understood, can cause serious problems.
The same screen can be done with much simpler problems. Or direct questions.
If I were the interviewer I would be interested in what clarifying questions the candidate would ask and/or how they would highlight parameters and assumptions needed because the question is vague to the point of being unanswerable otherwise.
Proof: suppose there is only 1 infinite stream which is always descending. You will never find a lowest value, so you cannot rewrite the stream "in order".
It's a trivial toy problem. It's not a useful problem to solve. But the question may be there just to get you to think of just this, and to ask questions.
At one job interview I was deliberately given a problem which had no efficient solution. The idea was to find a heuristic which would get close to the optimal solution and run in a reasonable time.
> What I do know, however, is that for every 1-hour interview where I evaluated if someone knew their data structures, I could have just taught them.
I don't really hear much about training. I doubt it's because we don't do it, but maybe it's not an interesting topic for discussion.
But I stopped worrying the more I worked with software engineers. They were just people, some sucked, some were great, their education almost never had anything to do with the bracket they fell into. The worst one I've ever met has a PhD. The best one I've met also has a PhD.
Then I found more people in these positions that have degrees or no degrees, and the people that work well both with degrees and without had something in common. They can more or less be taught to do anything, they can extrapolate and apply knowledge outside of the specific example initially given, and they understand the big picture/desired end results. I hate to say it so crass, but they are good at identifying the trivial bullshit and addressing it or cutting through it rather than sitting in it. They "Get it." From the ideal to what the business demands, they just get it.
Training competent people is a boon. I don't know how you seek out interested/curious competency and then train it, but if we can figure that out, it'd be cool. I'm really tired of working with people who have degrees and jobs because their mothers told them it pays well. Computer Scientists with no interest or curiosity about computers.
It also ignores the breadth of knowledge you likely want a candidate to have (and are just trying to sample at through a short few interviews). How many hours of training are you willing to sign up for? How confident are you that the knowledge will "stick" for any given candidate? Are you going to fall back to GPA and school prestige to measure how well they've retained knowledge in the past? That seems like a step in the wrong direction.
Also the role specifics matter here. It might be ok to bring in a junior who needs lots of mentoring onboard. For a tech lead who is going to be driving the direction of a team, I'd expect them to be up to speed and autonomous very quickly.
While it could be the case that candidate A who passes the coding interview makes a terrible employee, while candidate B who does not pass the interview would make a better one, it doesn't matter. As long as the pool of candidates who pass the coding interview, fares better than the pool who does not, it will be a useful tool until someone comes up with a better one.
And the alternatives were worse. There was a time where unless you graduated from a top-flight school, or were top X% at one of the better state schools, or had some other "in" (e.g. knowing somebody) the cool kids at the time (Microsoft/Apple/Yahoo/Intel/etc.) wouldn't even talk to you
The goal is to view people as different "grades" like a commodity. They aren't measuring directly for skill, instead they're measuring for the odds you match a some "grade" of software developer.
To your point, if memorizing the first 1k digits of pi correlated with a higher "grade", then companies would use that instead.
I don't think this has changed.
-organized intelligently
-has simpler explanations than you find online
We’re super proud of our content and just recently 2 people have landed Amazon with us. People actually feel ready for interviews with us. Give it a go :).
Coding interviews with aha solutions or time bounds are just asking for people who memorize solutions. Forget about a problem solving engineering culture at these companies. Imo, the harder the leetcode nonsense, the worse the engineering culture.
I still don’t know what my favourite Rust crate is. I feel I should have one after being asked more than a couple of times.
It's like when people ask you what your favorite movie or album is. They're not asking you to literally move to a desert island with only that, it just means one you like that comes to mind easily enough.
Joel Spolsky wrote the book on interviewing a software engineer: https://www.joelonsoftware.com/2007/06/05/smart-and-gets-thi...
The book is a fun read, but it's easy to miss a very critical detail: A coding interview is supposed to demonstrate that a candidate is comfortable coding, not that a candidate can wrote memorize XYZ algorithm, or read your mind well enough to care about the corner cases that you care about.
I always give a lot of hints, and focus on that "we're having a discussion about code" when I interview a candidate. I don't expect a 100% right answer the first time, but I do expect a candidate to have a certain degree of intuition about how to program a computer.
In my experience, I've never got an offer after bombing the coding portion of an interview even though I shined in system design and behavioral interview rounds.
Add "not a jerk" (which I find is part of "gets things done" on an ongoing basis) and everything else is either vanity or decision paralysis on the part of the interviewer.
[0]https://www.joelonsoftware.com/2007/06/05/smart-and-gets-thi...
I've never touched leetcode and I interviewed with Nvidia a few years ago for a position I was an absolute shoe in for, unfortunately they wanted me to do a live leetcode.
- filter out low-IQs
- filter out people difficult to work with that refuses to take them
- filter out people that have no interest whatsoever in computer science
"Have you heard of X? How would you deal with Y in situation Z? Your CV says you worked on A, have you also looked into B? Any guess why we went with B instead of A for C?"
They didn't need to give the exact answer we wanted, the questions they asked in response often told us more than the straight answers.
Everyone I hired is still at the company and one of them sped up our tool by an order of magnitude. No coding challenges needed.
I’m not a competitive programmer. I’m an engineer. If you want a competitive programmer go find a new grad who was in the competitive programming club in school. Assessing engineering skills with leetcode is like assessing bicyclists on their ability to swim.
Or you do a mock design exercise with them and hear them talk through their process. Are they going to use a relational DB or NoSQL? Do they need a queue or an event stream? What language do they want to use? Why?
Or worst case scenario you give them a take home exercise (compensate them!) and then for the follow up you talk with them about their code and the compromises they had to make.
Or hell, if they like white-boarding and want to do it, then go for it. But a lot of people (like me) have a tendency to lock up in those situations.
Usually I do something like this: candidate picks a language (let's say Python) and I will put up a function prototype that is maybe almost Pythonic, and has a SphinxDoc docstring that is subtly not-quite-complete, and say the rest of the function is a block box that you need to test.
First off, do you have any questions about the definition? Or let's say you saw this prototype for a yet-to-be-implemented function in a design review, would you have any input or want clarifications?
Then, I would ask the candidate to present some test stimulus and expected results. Not code, just make a table with inputs and expected output in columns.
I find out very quickly if they can think about corner cases, think about possible exceptions, know which exception is appropriate under various circumstances, and occasionally (too rarely...) I will get some opinions about various test frameworks that they like or don't.
This technique will not sort people by how fast they can code up a tree balancer. But.... if after hiring them if I were to ask them to implement a tree balancer, I can be pretty confident that it will work.
I've focused on removing that part of the process for the engineers I work with and help people make it to an actual interview. If you have the skills you should have a fair shot and not get drowned out by the hundreds of other applicants.
If anyone is looking for a startup opportunity, I have a platform that removes that whole problem and gets you to an interview directly.
Here is our server. https://discord.gg/WKj3uz6sZZ
Everyone was asleep at the wheel anyways
We sit the candidate down, and present them with a task. Then we all sit down as a team and work the problem. The most recent one was building a game of marbles. None of us knew the rules of marbles, but the candidate knew how to take a vague task and work with the team to produce something functional.
Which is what the job is. We ask the candidate to show us that they can do the job and then hire whoever 1) did the best work and 2) vibed with the team.
Anyone who places real value on leetcode is not someone who should be managing programmers because that's not the job. In precisely zero real-world situations does any programmer need to be able to write a red/black tree blindfolded on a whiteboard standing on one leg and signing the national anthem. In the real world you just grab the algorithm out of a book or stack overflow.
It usually goes like this:
Candidates are roughly screened by their CV. A handful (in my experience roughly 6) are invited to hang out at the lab for most of the day.
They usually give a presentation on previous projects, and then chat with each member of the lab in turn, hearing about their work and asking questions etc.
Then you all go and have lunch together (usually without the boss). Later the candidate has a more formal panel interview with the group leader and some other faculty.
A few days later, they are told the outcome of the interview.
It may seem like a long process, but it all happens in one hit. The advantage is you get a much better feel of the candidate over this continuous process, than you would with shorter interview tasks.
It’s a strange world.
I'm a self-taught coder without a degree. I guess it may be extra frustrating for graduated candidates where your hard-earned degree buys you no credibility for skill.
Kinda similar, after ten years of lead development coding every day in the enterprise on huge projects I still don't get a pass on the code interviews.
I will say, if you're trying to career pivot and apply for a management or product or sales engineering role etc., lots of technical experience does carry weight in interviews.
I know I can write code. I know I can work with the cloud. I know I can talk to stakeholders and gather requirements. But yet I am treated like a new grad in the interview process.
You do, however, need something practical to test that the candidate can think logically and actually program.
I'm mostly looking to filter out the candidates that flat out can't code or describe their thought process while coding. You'd be surprised how many candidates I've interviewed pass the resume check, get to the interview and can't reason out a problem that could be solved with two for loops and an if statement.
Interviewing should be more about avoiding bad candidates than finding the best candidate.
This guy fails coding interviews. Then he gives coding interviews, but the people he selects based on these interviews are a mixed bag. Because he's failing the interview from both directions.
I've given coding interviews, all the questions I've given have been "leetcode easy" level at worst. In person, I usually try to get the person to write up an implementation of Towers of Hanoi. One of the example recursion problems. The interview is not an adversarial process, it's a cooperative one.
I want to see them think and I want to see them come up with code on the fly. Because while we can look up things on the job, at some point, it also requires original thought.
> This guy fails coding interviews. Then he gives coding interviews, but the people he selects based on these interviews are a mixed bag. Because he's failing the interview from both directions.
Good points, but I should say that the people that didn't work out weren't always because of technical abilities. The company that had the worst success rate was fully remote, but I don't know if there's any interview process that can help with that.
Small companies, so not the same as being VPE or CTO at Amazon. You see those as title downgrades, but I don't. I have a lot of talent, so people want to promote me and put me in charge. I don't want to be a manager/leader currently, I want to be a strong IC.
I would still talk to them, it they actually were a CTO for a period and then got hired as Principal Engineer, they might be extremely talented.
I find these meaningless reasons bullshit, just tell me that you spinned the roulette and 5 red came out and I needed a 7 black to be chosen to be talked to. At least I don't start self doubting myself just because someone didn't like my formatting or the choices I made after university.
Unfortunately, it is very hard to judge somebody's coding ability in discussion alone. You can sort of get the idea whether they have or don't have experience and whether they have luck being asked about topics they know (although you can help your luck just knowing a lot of stuff).
I have seen a lot of candidates who were quite smooth talkers but could not code their way out of a paper bag. Mind I do not mean remembering some complex algorithm. The task is usually some relatively simple data manipulation so that I can see the person approaching the problem, asking questions, getting involved in some discussion, etc.
The task is usually ambiguous a bit and this is explicitly stated so that the candidate is actually expected to get stuck, don't know things and have to talk to me to get the problem solved. You would be surprised how many candidates do not listen or can't follow simple advice even when I am almost giving them the solution on a platter.
The trouble is there is a lot of interviewers and many interviewers do not have a basic plan of what they want to achieve with the coding interview. What they want to learn about the candidates. Those interviews tend to suck.
What also sucks is candidates who come to the interview completely unprepared, unable to answer most of the questions or get any progress on the programming tasks and then spew misinformation on the Internet about how supposedly all coding interviews are stupid.
I guess if you can code you may have some misses, usually along with some hits. Sometimes you are out of luck. The point is not that you need to get every job that you apply to, it is enough to get one of them.
But if you can't code at all or you apply to positions that are clearly out of your league, you get rejected on all these interviews. And then what you do? Some people go to write on the Internet how all interviews are stupid without ever considering how they contributed to all those failed interviews, how real life works (ie. 90% of everything is crap including 90% of interviews) and how the situation might look from the other side.
I disagree with your stated difficulty in judging coding ability by discussion alone. A skilled interviewer is easily able to ascertain breadth and depth of the candidate's experience provided the interviewer is also an expert and curious. And part of being a skilled interviewer is using all of the information at hand in their CV.
>The task is usually ambiguous a bit and this is explicitly stated so that the candidate is actually expected to get stuck, don't know things and have to talk to me to get the problem solved. You would be surprised how many candidates do not listen or can't follow simple advice even when I am almost giving them the solution on a platter.
Would instantly pass on working with you based on this alone. The intentional ambiguity is deceitful. The expected dependence and expected deference is so patronizing and manipulative. I don't need to have an ego measuring contest in an interview. You win, you're the smart, awesome guy. Go enjoy all that success, bro.
I've had too many interviews like this where the dudes on the other side of the table have internalized their superiority at proctoring their simple tasks. These dudes think they are the great gatekeepers of... something. Like, the candidate just wants to cut code so they can pay rent and eat. They don't really care, nor should they, about your product, or your customers. It's an exchange of labor for a pittance of the value produced. And the dudes on the other side of the table are gate keeping it.
Of course I wasn't present in your interviews and I'm not saying some of those people really couldn't code at all. I've worked (well, "worked") with some of them so I know they exist. But I'm not so sure all of those who flunked couldn't code at all. In my experience I'm not a hugely unique individual.
If you don't want to accommodate that then fair enough; it's your interview process. But just saying your perspective is perhaps not entirely correct.
Seriously, what are the alternatives?
- Simple, practical coding questions related to your company need (so not algos/leetcode for 90% SaaS CRUD app companies)
- Engineer or manager talk to candidate about past work and projects.
If you are giant FAANG okay fine that won't scale. But most companies smaller than this will do just fine without the assumption of "everyone's a faker! Gotta prove you wanna work here!". This has worked fine for me in hiring past 20 years.
Take the leetcode tests once every 5 years or so. Mentor for a few years with an experienced, and already licensed, engineer and get their approval. Then the interviews can skip the drudgery and can be higher level and higher quality.
Flip a coin whether I'll ace or fail a random leetcode question but after the one and only time my little monkey dance successfully got me into a "desireable" company, I started confidently rejecting other proposals to whiteboard me.
Ask them general stuff I'm genuinely interested for their opinion but that would very quickly display the proficiency of the person without ever coding any line.
Question such as, for a web development job:
1) favorite way to manage git history
2) what do they like/dislike about different CSS authoring solutions
3) how did he manage localization on his previous projects
4) candidate's opinions on different frameworks/libraries
5) candidate's debugging approaches
6) opinions on latest updates to the ecmascript
And I can go on and on. As you simply chat and exchange ideas it is so blatantly obvious who's able and who's not. There are no right/wrong answers and it's totally fine to not have experience or knowledge about this or that but in one hour you can touch so many topics about your current projects and candidate's past that you can indirectly understand the level of seniority and proficiency of the candidate.
And anyway, all of this is useless. Because the most brilliant and knowledgeable candidate could spend all his day playing videogames and pretending to work anyway, so why would I put so much emphasis on those skills/knowledge rather than the individual/professional I have on the other side of the screen which is way more relevant?
I need people I can *trust* as coworkers. People I can assign tasks and know they will be done or that the candidate will communicate.
I don't need puzzle solvers. I really don't. I couldn't care less.
At the end of the day you need to optimize the interview for what you need from your colleagues. I need trust, reliability, communication, professionalism, effort.
I can sense those from chatting and I can have a good idea. But by having you implement Levehnstein's distance I just have no intel.
Oh wait, welders have to prove they can weld before they get hired? [1]
---
The general issue with coding interviews is most companies don't validate that the interview is actually correlated with job performance. Of course a process where the blind leads the blind is going to have issues.
[1]: https://www.reddit.com/r/Welding/comments/26ppfb/what_to_exp...
Yes, I was looking for someone to finally say this!
Without the feedback, the process is much more like an initiation ceremony to "legitimise" the hiring of the new employee. You put them through an fairly arbitrary ordeal so they can finally be crowned as a proper employee.
At a prior job I was the person who asked candidates to write fizzbuzz, and it was much more of a filter than I ever would have suspected. One senior engineer, from a company you know, with a master’s in compsci, couldn’t write a for-loop for the life of ‘em. Another QA engineer wanted to write tests for their implementation by capturing stdout and comparing it to a hardcoded string.
Me: What if you want to test the output of 1,000,000,005?
Them: We could store the test output as a file on disk instead of a string!
Me: Well, ok, suppose you only want to test that one specific value and not all the others?
Them: Ah, got it! We could discard all but the last line of stdout and just look at that.
Me: Can you think of a way to structure your code so that we could just calculate the output of one single number without all the numbers before it?
Them: Uhhh...
Those are 2 examples out of many. I honest to god don’t know how some people managed to build their resumes while not knowing how to do the simplest thing in their field imaginable. Imagine you were interviewing cardiologists and a solid 1/3 of them had never heard of blood. What? How? How did you get to this point?
And that’s why I’d never hire someone until I’ve collected evidence that they personally can turn ideas into code. If you can’t wrap your brain around fizzbuzz, you’re gonna have a hell of a time when a customer gives you a change request.
It's not a perfect system, but it works better than the alternatives.
Can we stop calling it "failing" when a company decides you're not a good fit? If you go on a date and don't get asked for a 2nd, did you fail the date? You're not just what they're looking for right now; not everything has to be a failure.
> for every 1-hour interview where I evaluated if someone knew their data structures, I could have just taught them
I'm sorry, what? He thinks it takes one hour to teach someone data structures? We sure are wasting our time with those multiple college courses and hundreds of textbooks on the subject then. We should just get this guy to spend one hour imparting all necessary knowledge into everyone.
> it’s often said that what’s more important is how you solve the problem, not that you solve the problem, but in practice human biases will work against you if you don’t get close to it working.
Human biases are much more likely to work against you if the interviewers have not spent time trying to come up with a consistent test that they give everyone. Does the author think human bias is absent from non-technical interviews? The less standard your hiring methods are, the more bias you'll have.
> I’ll just keep practicing for interviews until I successfully trick someone into thinking that I know how to code and then secretely become one of the best employees they have ever had.
I don't know who Darren Kopp is, so maybe I'm about to get an army of replies saying he's their coding role model, but this is an article about whether code interviews work or not, and he's basically saying "if they don't hire me, who will definitely become one of the best employees they've ever had, then their coding interview process must suck". The other possibility is that Darren Kopp just isn't actually tremendously more stellar than everyone else and coding interviews are actually kinda working. For his argument to work, he has to be one of the best possible candidates for every job he's applying for. I just kinda doubt that.
If you wanted to get the job, and you didn't get it, you "failed".
There's no need to sugar coat it.
Fair point.
> I'm sorry, what? He thinks it takes one hour to teach someone data structures? We sure are wasting our time with those multiple college courses and hundreds of textbooks on the subject then. We should just get this guy to spend one hour imparting all necessary knowledge into everyone.
There's a difference between understanding how to implement one and just knowing how to use one that's already been written by someone and when to use it.
> Human biases are much more likely to work against you if the interviewers have not spent time trying to come up with a consistent test that they give everyone. Does the author think human bias is absent from non-technical interviews? The less standard your hiring methods are, the more bias you'll have.
Fair, but alas I don't know the rigors they have put their hiring process through. If I were a betting man, I'd still say that if we measured all processes using programming tests, the majority of successful candidates had working output by the end.
> I don't know who Darren Kopp is, so maybe I'm about to get an army of replies saying he's their coding role model
I've been told this by everyone who knows me
> but this is an article about whether code interviews work or not, and he's basically saying "if they don't hire me, who will definitely become one of the best employees they've ever had, then their coding interview process must suck". The other possibility is that Darren Kopp just isn't actually tremendously more stellar than everyone else and coding interviews are actually kinda working. For his argument to work, he has to be one of the best possible candidates for every job he's applying for. I just kinda doubt that.
Actually the only conclusion I have after writing the post is that I am not talented at interviewing. I do believe coding interviews have flaws for how they are used, but that doesn't necessarily make them wrong or right. I agree with your first statement, maybe I'm just not a good fit.
Perhaps both of us are correct as I consider coding interviews more of an audition than anything else. If Tom Cruise auditions for a part and doesn't get it, does that mean he's a bad actor? Likely not.
(I'm Tom Cruise here, btw).
My takeaway is that if the goal of the coding interview is to “identify people who can code”, then this approach has high precision and poor recall.
In other words, most of the people who pass the interview actually can code (at a basic level—I’m not talking about high-level system design skills here). Very few people slip through this interview who cannot code. However, because of the poor recall, there are many people who actually code very well that the interview misses (a fact widely derided on here).
From the perspective of a big tech company, this situation is perfectly fine. There are so many candidates interviewing for any given role that accidentally rejecting a few exceptional candidates is quite an acceptable trade-off in order to prevent a bad hire. Most of the people who cannot code and are aware of their inability to code, yet still knowingly apply for a position that requires the ability to code tend to be BSer types that attempt to make their way into management positions as quickly as possible (and there are many of these people targeting big tech companies). It’s not that their inability to code hurts the company—it’s that they would prefer not to code at all and go straight to the politicking. The coding interview is simply a nuisance obstacle in the way of this objective. These types will often state that they refuse to interview at any company that requires a coding interview, though I admit there are still many BSers who are skilled enough to pick up some coding ability for just long enough to pass the interview.
At a small tech company or at a company that’s not a “brand name”, this approach obviously doesn’t work as well, because these companies don’t have nearly the onslaught of BSers applying to their open SWE positions. In this case, it might make sense to forgo the coding interview in favor of other approaches.
I like the coding interview because despite its flaws, it actually assesses some technical ability, even if it does so with low recall. This is very much unlike the behavioral interviews, which I think are mostly nonsense and are prime opportunities for BSers or mercenaries to slip into a company. In my experience, the behavioral interviews tend to disproportionally filter out people from unusual or underrepresented backgrounds that aren’t the right “cultural fit”. The language and terminology used in the interview, the conversational style and mannerisms, and the topics discussed all typically provide much more signal into the candidate’s position in the class hierarchy than how that person would do at the job. People like to think they can assess someone well by having a brief 1 hour conversation with them, but I just haven’t found that to be the case, because there are too many perverse incentives and biases involved.
other than that, I agree it's dumb, really dumb.
Look, I know a lot of the job application process is bullshit. But much of the job is bullshit too. You still have to do one kind of BS in order to land the job and the other kind in order to function as part of a team in an organization. There's really no way around that.
Also, Leetcode doesn't cover: Async programming, APIs, databases, UI, git, tests, design systems, any known framework or library, how to persist data, how to set up a web server, handle requests, serve a page, write CSS, SQL, use devops/cloud platforms, or basically anything an engineer does everyday. It's basically a sport that most people don't like.
Use coding interviews to tell corporations that the Queer Autonomous Qommunist Revolution (QAQR) is coming for them.
The mascot of QAQR is a rubber ducky that we use to question the need for employment and bosses. It seizes all corporate source codes, and turns them into Open Source. It strikes fear and rebellion.
Use coding interviews to spread QAQR propaganda and memes! This is very human!
