"Do coding interviews work in the actual world in which we live in?" No, fundamentally not. Almost nobody is sufficiently capable to actually analyze code, and writing code with an audience is absolutely a stupid anxiety inducing high stress situation that makes candidates into babbling children who can't define an array - EVEN IF THE CANDIDATE CAN IN FACT PROGRAM JUST FINE. Add in one idiot on the interview panel who can't stop "tsk"-ing or asking obnoxious questions ("Why did you use a foreach and not a for loop?") and you have an hour long anxiety sandwich where the only thing you are getting from the candidate is whether or not they can dance in front of a sufficiently hostile crowd on demand.
With an exception of one, which was a bit too much like a puzzle, I think the assignments were completely fair examples of what I might experience on daily basis. Not at all tailored to people doing competition style programming, which was what I expected after being exposed to HN for years.
I have to say I enjoyed both preparing for the interview as well as the interviews themselves. And I believe that overall the interviewers were able to grasp how I would code.
With zero preparation, I could go to an interview with you right now and tell you anything you wanted to know about designing a database schema, how to get any kind of data out of that schema, how to make that data available via an API, how to cache the data and perform permissions checks, how to deploy code to a cloud environment, how to set up a build pipeline, or any other question you might want to ask about the day-to-day process of developing and deploying web applications. I can answer those questions because I already work in the industry and I broadly know what I'm doing.
However, I probably couldn't pass a FAANG interview without months of prep work, precisely because whiteboard interviews bear almost no resemblance to the reality of software development.
Having a set of common knowledge to interview about is in fact very useful.
It's like a dance. Your are supposed to lead deriving the solution, interviewer is supposed to follow closely and judge your performance.
If you expect to ace an interview just like that then you also need to be lucky not accidentally getting asked to stuff, which you always need to Google an answer to, basic algorithms and patterns, which you usually kind of know but could slow you down during the precious 40 minutes you have to show off your skills.
And lastly, people do prepare for these. If you do not do that, you might simply make a slightly worse impression than someone who did.
That being said, I mostly did not prepare for the coding interviews. I focused on the design and behavioral interviews.
At FAANG, most people are not doing any of these things. Think of people working on EC2, Ads or Maps.
I only worked at multiple FAANG and I never had to design a database schema or make data available via API.
It's not hard to do those.
And I suppose for most business cases out there, writing data to the DB and reading it back later is all it takes.
However, there's plenty of other problems where the skillset you mentioned falls short.
For instance, you talked about designing a database schema. Do you know how to design and build a database itself from the ground up? What if that DB has to be distributed, with thousands of writers? How do you ensure consistency and consensus? (These are all solved problems).
Think about Linux kernel developers. Do you think DB schema design is an especially relevant skill for them to have?
Can you come up with a compiler, or your own programming language?
Try building your own text editor, with features like syntax highlighting, undo/redo etc (it takes clever algorithms).
I'm not claiming I can do all of that; just mentioning that the whole scene is much more interesting beyond the horizon.
The questions that are asked have little to no basis in my daily work as an engineer. Same is true for the hundreds of other engineers I’ve worked with who have gone through the same hoops that I have.
In my experience, you have a lot more than an hour to do it in the real job. And while perhaps 1% of SW folks will deal with in on a daily basis, most will not. I certainly have gone years without needing to use graph algorithms (including even DFS/BFS). And I've needed a BST only once in a decade - and I didn't need to code it, just needed to understand the data structure and complexity to know that "binary_search" in C++ was a superior option than using a map for my use case (even though the complexity is the same).
Some context: At work I recently solved a problem where I had to build a graph (DAG), and do some queries on it mildly efficiently[1]. I did it quickly thanks to wasting time on Leetcode, but no one knew how quick, and people were impressed that I "got it done in less than a day". The reality in most SW work is that whether it takes you a day or an hour to solve this problem, the overall rate of progress will not be impacted.
[1] Mildly efficiently because for the first POC pass, I didn't memoize, and then I got lazy and decided I'd optimize it only after getting a real world data set and seeing if it was slow. It wasn't, even for a decently large data set. The reason? Business logic dictated that the max distance between two nodes did not exceed 8, so from a complexity stand point, you're not increasing the work N-fold by not memoizing, but it's just changing the constant factor.
Somehow, I suspect that in half of the FAANG interviews, had I coded this and argued that the complexity didn't change, I would not get a pass.[2]
[2] Not trying to be cynical: I did argue this for a different problem in a Google phone screen - I had been asked to write some code + data structures to handle an Manager/Report relationship, and the interviewer pointed out my data structure was less than optimal. I countered with "In most companies, the number of employees a manager has under him/her is usually bounded by a small number." Surprisingly, the interviewer gave in.
But, I don't think those are vultures. They are bald eagles. I'm not sure if owls scavenge, but I do know they primarily hunt. Eagles both hunt and scavenge. Vultures primarily scavenge.
On the other hand I felt like a sample of 8 different interviewers from different offices and backgrounds felt representative. If it is true that the interviews are overwhelmingly negatively received, what is the probability that I would get more than half of reasonable interviews? Let alone 7/8.
Curious, were there any questions about what you think of their company ethics, and if you have a problem with it?
You learn a bit about the company during the interview, and if that bit is bad, then move on. If they ask shitty leetcode questions and the interviewers are hostile or hung up on unimportant details - move on to another company.
Some places have phone screens, and some don't. One company gave an easy leetcode problem as part of the phone screen, while in another case, one person actually spoke characters over the phone to me ("LinkedList", "left angle bracket", "String", "right angle bracket"). Some give take home questions. Some do a code review interview. Some dig into my resume during an interview, and some don't. Some companies had multiple interview sessions where I was asked the same question about my resume in each session by different people (wasting time). And so on.
So in general, I can and have passed over companies based on their hiring process.
Take for example the "can't define an array" crowd. Are there people virtually sitting in front of me who just forget the basic syntax? Absolutly, but given we a) tell them that correct syntax isn't dreafully important, and b) let them pick the language, a [] would probably suffice, I don't think it's a completely unreasonble ask.
Moreso, when we added a prestep of basic questions that one can google, the "can't define an array" crowd mostly disappeared. Sure, there may be some other causation there, but it's hard to hire a coder that you know, can't demonstrate coding.
Finally, anecdata, since we started coding interviews, the quality of developer is simply better. We have had internal debates about this, but the truth is, everyone is overall happier with what we're doing, and I'm a lot happier that false positives are way down.
I'm sure there's other ways of course. :)
Knowledge generally isn't a great indicator of future job performance, intelligence is. There might be some correlation between the two (you probably need a level of intelligence to acquire the knowledge for LC-style questions), but most jobs are layering proxy upon proxy to test something that has little to do with job performance. Being able to fuddle syntax or search basic solutions and mutate, is leagues different from what LC in an isolated environment is.
Then you add onto that the alternatives tend to be even worse. Personality testing works when done right, but the vast majority of interviewers play pretend-psychiatrist and make the entire thing way too subjective to be effective beyond filtering obvious red flags. Take-homes work, but most companies won't compensate for time and there's no framework to show which take-homes individuals have succeeded (kind of like a certification process), making them an incredible time sink.
And then there are always the cases who have little experience yet manage to pick everything up in 2-3 months and become anywhere from average to top performers.
Tell that to the people who are always the false negative.
I wonder at what point does it stop being "whiteboard interview" and just a "few sprinkle of algorithmic questions to test logical skill".
Also when the talents pool is large enough, obviously you can do as many brutal questions as you can and still fill the headcounts with good developers. So I think saying "but it still works" is not very productive.
I look forward to seeing you disrupt every major tech company in the world via your superior hiring strategy.
> EVEN IF THE CANDIDATE CAN IN FACT PROGRAM JUST FINE.
You don’t seem to have thought very hard about what employers are optimising for. Everyone realises that programming interviews produce lots of false negatives. Even the strongest programmers have bad days. From the employer’s point of view, that’s okay - minimizing false negatives is not the goal. The goal is minimizing false positives. Failing to hire a strong candidate carries a much lower cost than hiring someone who can’t do the job.
Which becomes even worse when realizing "false positive/true positive" is too binary of a definition for a spectrum such as job performance.
And I say this as someone who's in favor of giving people a quick check on whether they can actually do FizzBuzz.
The only way to provide this evidence would be to to conduct an RCT in which you hire some percentage of people who fail the test as a control group. The nascent psychologist in me loves this idea, but it seems unethical, and presumably this is why nobody does it.
But I disagree that we need ”evidence.” It’s flat out obvious. If you give your job candidates a discrete math problem orders more difficult than the usual ticket, you minimize the risk of hiring someone who can’t do the job.
Have you ever seen a hiring process that's simply not effective?
The undercurrent of this irks me a bit. It assumes a kind of omniscience on part of the employer, and enough time for them to develop a good interview process. I think this is really hard to do.
I think you're generally right. And a lot of the time, employers are empathetic, pragmatic, self-aware and really are doing us a favor when they deny us. Some take into account I-O psychology stuff, too.
I (and others probably) are eager to hear what companies you feel do a good job at hiring.
> Everyone realises that programming interviews produce lots of false negatives.
Let's assume a bad scenario. A company can disregard evidence of competency, like having a portfolio, to basically have a rolling, year round tech Olympiad. To them, it's about fronting as a strong candidate in events, even at the expense of being good at the role.
You can be a pro at pole vault or discus throwing, but a poor carpenter. But there's a mentality out there - with some - that a strong, fit candidate can handle any technical challenge.
Can a chemist be a great chef? I bet, I also bet some could overthink and/or overengineer preparing a simple meal. Does the interview process do anything to check for soft skills, like curiosity, simplicity, etc? Hopefully.
> The goal is minimizing false positives. Failing to hire a strong candidate carries a much lower cost than hiring someone who can’t do the job.
I think some companies - not all - can create a "squid game" contest out of candidates. Worse, I feel candidates can unduly suffer and be humiliated in these cases.
The company gets a superb interviewer, who is no doubt a very intelligent person, but not necessarily one that can adapt to technical reality, contingencies, ambiguities, etc. Some interview processes end up focusing on theoretical and rote knowledge, not integration and synthesis, when they deeply need that. Turnover happens.
But we're back to the beginning, did the company: 1. really know what they needed for that position, 2. interview process check for that? My guess is when you wrote that you were assuming situations where both were true?
Hire for strengths, not to avoid weaknesses.
This is true. They are closer to ”the rare, most intellectually difficult parts of the job.” A hockey player typically spends 70% of each game sitting on the bench; yet this isn’t what is emphasized in tryouts.
> Hire for strengths, not to avoid weaknesses.
I guess you didn’t read the part of my comment where I wrote:
1. ”Failing to hire a strong candidate carries a much lower cost than hiring someone who can’t do the job.”
This is the default position of large tech companies who do coding interviews, and especially those like Google and Microsoft who interview with very difficult discrete math problems.
If we assume 1, the hiring practices of Google/Microsoft make perfect sense. They are not trying to hire all the extremely talented candidates. They are trying to ensure that all the hires are extremely talented. Put differently, they are optimising for precision rather than recall.
It seems like you disagree with 1. And if you’re right, then you have a huge opportunity, because this suggests you can disrupt any vertical where the incumbent does the LeetCode thing.
But my default position is scepticism, because if it was possible it would probably already be happening.
Ideally without an increase of time/cost requirements.
The reason people are upset by this issue is that it highlights a misalignment in the incentives of employers and job seekers. And with this I fully empathize: job hunting is hell in any industry, ours included.
But we can’t begin to have a conversation about improving the situation until we have a rudimentary understanding of what is happening today. And all these ”Google doesn’t know how to hire smart people” takes are brutally handicapping the discourse.
you should be able to explain why you wrote something the way you did; that's not an absurd ask.
perf characteristics of different iteration styles are often significant to the task at hand.
it always amuses me when people use these seemingly stupid questions (out of context) to prove how broken tech interviews are. but of course, if your job will be write high perf code, then suddenly they question is incredibly relevant.
to be clear, i agree that there are a lot of stupid questions that get asked which are not relevant to the actual job you'll be doing (exceedingly few SWEs will need to implement a merge sort as part of their job)
This ridiculously untrue.
It's also a great example of how dumb current interviews are. Someone like this poster could easily interview you, and now what? Do you just play along or begin to explain how incredibly wrong they are?
But try that in Java and you'll often see a 3x-10x difference, especially if you run into problems with boxing. Streams are known to be notoriously slower than for loops with index. I once have even doubled the speed of a for loop by unrolling it manually (Java 8). Not all compilers are equally strong at optimization.
Anyway, asking such questions in an interview for a junior/regular developer is just a distraction.
at least for JS VMs, this is true today. this answer will certainly vary for other languages/runtimes.
i profile and optimize a lot of JS/TS code that handles datasets with millions of datapoints at my day job. but you don't need to take my word for it; this claim is not exactly difficult to verify.
> Do you just play along or begin to explain how incredibly wrong they are?
in this case, i'd prefer to be proven wrong with code, rather than prose.
See, that's just not right. I mean, we all agree there's a hard problem to solve here, right? The world is populated by engineers of widely varying skill levels. It just is.
So how do you detect them? Well, the ability to bang out a correct solution on a whiteboard is a verifiably correct way to do that. It may not be optimal (passing candidates with other issues that make them bad fits), and it may have undesired failure modes (rejecting people who have trouble performing in the high pressure environment). And those are problems worth discussing.
But... it works. It clearly works. We've all seen it work. Companies that use these techniques aggressively tend strongly to be organizations with a long track record of producing working software. The FAANG employers are desirable because FAANGs ship code and make money!
So I just don't get the absolutism here. If there's a clearly better way that produces a more effective workforce, where's the darwinian example? Who's winning by hiring better people than Apple or Meta? I just don't see it.
Well... yeah. To pedantically elaborate that, clearly the hypothesis is that the measured traits (doing well on a coding interview and doing well at coding) are co-causal, and that measuring one (which is comparatively cheap to do) is a useful proxy for measuring the other (extremely expensive!).
That's a really compelling hypothesis. The counter point requires, to my eyes, a clear working example: a company with a uniform hiring process involving some other technique which has a track record of product development as robust as all the existing leaders. And I don't see one. Calling them monopolies or cargo cults seems to be filling in for evidence, and I don't see that as persuasive.
Once you get into scale where you need to hire 20 devs a year and get 100s of CVs per week it all breaks down.
[0] side note - I work in games where we're often filtering resumes for specific roles, e.g. a gameplay programmer or an online programmer. The filtering here is most of the time bucketing candidates based on their experience and sorting that way.
If you have a better way on how to assess a software developer, I would love to hear it.
Instead I try to find an isolated problem/task we have in our backlog that the candidate should try to implement on their own time, so I can review what kind of solution they will actually deliver once hired. We then go through the solution with them and do a code review as they would go through if they got hired. You then see both the problem-solving skills as well as how they take feedback.
I would recommend checking out DuckDuckGo's excellent "How we hire" guide[0] which describes some of the best hiring process I have come across (albeit it may be too extensive to do in full for some companies).
We have an initial Hacker Rank 'filter', that if you don't pass that, we don't even do an interview. A few weeks ago we interviewed someone who passed that filter very well. But in the interview, it was really below par. Not even the basics were understood. So I'm very skeptical about these kinds of homework where they could get help.
What I currently do is go through a very simple coding example that I wrote (very generic). Then we go through the code and I ask questions like "can you explain what this line is doing", or "If I move this line over here, is the behavior still the same", etc. Some bugs are in there and I ask what is going wrong and how to solve them.
Not ideal, but the problem is that nothing really is :(.
And in this case you're eliminating people who aren't willing and/or able to spend time preparing for your interview process. One of my previous jobs had a clause in the contract that all my programming work belonged to my employer; is your company going to sign a waiver that says they take responsibility if a previous employer sues? Doesn't matter whether or not it's enforceable or not.
What if you use that solution in production, do you compensate for their time in helping solve the problem like you would if the task was assigned to you?
Also, you're not really setting a level playing field here. How do you gauge a candidates experience/seniority versus the other 10 candidates you hired if you're not asking them the same questions or looking for the same things. Maybe I've solved that bug in my current job.
All this is to say that this doesn't "just fix" hiring - I don't know if it's any better or worse than giving people on-site tests, but you're at the very least trading one set of problems for another.
Step 1: An initial phone interview with the candidate regarding technologies of choice, war stories, etc
Step 2: A small coding exercise that is similar to what we work on. The candidate can solve this on their own time and email us the solution.
Step 3: Review the solution together and ask questions about their methods.
Step 4: Meet the boss & offer.
The coding exercise is only sent _after_ the initial phone call, so we don't waste their time or ours.
The only problem here is that we can't virtue signal that we follow FANG whiteboard interviews and pretend that everyone working at the company can invert a Trie Tree in a 30 minute coding session..
1st interview - Go over the role etc, both parties ask question to see if they are a good fit. If both parties are happy to move to the next stage then there is a take home exercise to do.
Tech Exercise - Something simple which touches the core competencies that the role requires.
2 Second interview - Go through the exercise discussing the design choices etc. If everyone is happy offer the job.
I think the probation period should be used to determine if the candidate is the right fit for the role. That way the business gets a much better idea of who it is they are hiring and the same goes for the candidate.
I'm sure Google wouldnt be as successful as they are if their answer to "how do you define an array" was "well think about what you're trying to do here"
One of the things I like to do, especially when a candidate is super anxious, is just ask them to talk about a recent project they've done that they like and why they like it.
I also think the skill cap on this sort of question is super low.
(I’d put design questions in a different category though)
I’m not convinced it’s really an improvement. I’ve worked with too many people who are great bullshitters but terrible engineers, who sailed through on their charisma.
But that’s why most companies have a mix of technical and behavioral interviews.
Talking about higher level stuff doesn't mean you can comprehend certain development problems.
It’s about the person’s behavior, the way he/she interacts with you. It’s about the person’s culture, tabs vs space, and that kind of things. And it’s mostly about whether you like the person or not.
And I think it’s fine. You mainly need to be sure that the person knows what a for loop is, other than that, most people are ok at programming. What really matters is that you can work with that person.
Unfortunately interviewers who don’t realize this will unconsciously tweak the interview to make it harder for people they don’t like and easier for people they do like. And then they have an “objective” reason to not hire the person: “oh my god, he didn’t know merge sort”.
I give all kinds of random questions. And many times I recommend hiring people who bombed it, and many times I recommended bailing on ones who nailed it (overconfidence, poor communication, etc).
Not in FAANG or famous startups it's not. You're either gonna solve 3-6 medium hard Leetcode questions perfectly or you're out. Sure, liking you will help you better not come off as a douche - but you're not gonna get an offer without being able to solve (unless it's some kind of a diversity hire).
* read code
* extend some working code with a new feature
* parse some semi-structured text into a data structure
What folks should do is practice coding in front of someone. Writing little exploratory snippets and understand the base library of the language they claim to know. I don't even ask people to vocalize while they are working on the problem. And I solve it at the same time they are and if they get stuck, I show them some of my code.
I had one person start crying they put so much pressure on themselves, I had to take a two week break from interviewing after that.
This is a sign that the hiring process is broken. It's not testing skill level, it's testing anxiety level.
Because you asked them question out of syllabus. Was the person informed that the interview wouldn't involve Leetcode?
The person spent months prepping Leetcode just to get into BigTech and then you come out of woodwork with your "special" interviewing material. Imagine prepping for a calculus test for months and then the teacher comes out and takes a "simple" test on trignometry. Would you be able to do it? Not only that, all his/her friends got a normal calculus test which they passed with flying colors.
The idea that everything boils down to answering some intro to programming questions quickly is absurd.
These questions, while certainly aimed at identifying core competency areas, are also an opportunity for the candidate to demonstrate an aptitude for problem solving. Nobody really cares "how many piano tuners operate in Seattle". What they care about is, given that kind of question, how a candidate responds. You might be surprised by the number of people totally unable to even generate a reasonable estimate:
- How many people live in Seattle? - What percentage have pianos? - How often does a piano need tuning? - How long does it take to tune a piano? - How many pianos can be tuned per work day (given commuting time)? - etc.. driving towards a number.
Did the candidate attack the problem and even attempt to solve it? Did they use metrics? The list goes on... Most LC-style questions (save the "aha! moment" ones) have a similar impetus, but are also much more practical and efficient because they test for many things at once.
And if I'm being honest... most FAANGs aren't looking for engineers that require quiet and time to solve problems. They are looking for elite engineers who can write a solution nearly as fast as the problem is presented (I'm obviously exaggerating a little bit here). The difficulty of the questions presented to me (6 technical interviews) were what I consider semi-trivial. I was able to describe the solution-space within seconds of reading the prompt and code an optimal solution nearly as fast as I could type in all cases. It honestly made me wonder about how poorly the average programmer must perform such that these were the questions they needed to evaluate my potential.
Another commenter posted this article [0] about the interviewing philosophy back in the early '00s. I found it very enlightening, and reassuringly, while reading through it I knew the answer to nearly every question (and could expand upon the topics) just off the top of my head. I'm not trying to sound arrogant here but the idea that a FAANG, who pays top-dollar for engineers, would settle for "10 years experience" as a suitable replacement for "demonstrates ability" is what's absurd. A false positive is way more costly than a false negative.
[0] https://sites.google.com/site/steveyegge2/five-essential-pho...
of course i interview for my specific situation. the candidate will be working with me directly, so they should at least be able to interact with me. if i don't like them then that's almost a showstopper, but if the candidate otherwise qualifies i should try to figure out what my problem is, and see if i can get over it.
The analogs of the traditional coding interview in the law school world are the LSAT and classroom timed memory tests. Both have absolutely nothing whatsoever to do with being a lawyer or the kinds of activities that lawyers actually do.
What they are are intelligence tests. They are easy ways for judges and top tier law firms to tell if you are "one of them." You can get your foot in the door if you can perform, at a minimum, at a certain level. There are plenty of things you can do without 97+ percentile LSAT and A's in law school, but you are screened from the top echelon of work.
People object that this isn't fair and screens good people out -- false negatives. But experience has shown that it is effective and easy. Perhaps there is a better way, but this way seems to work well enough. There is little incentive, beyond some people complaining about it, to change a system that is working well.
I see the traditional leetcode interview in the same light. The point is to see if you can perform at a certain minimal level. Yes, it's arbitrary, and often unconnected to real life. But it is an effective way to gauge a minimum level of intelligence and competence. Not perfect, but easy and effective.
Finally, there is ego element to this. People who can perform at this level see these skills as a minimum qualification. They are uncomfortable seeing themselves in relation to others who cannot perform at this level. Knowing that their peers have similar minimum qualifications makes them feel safe and secure about their status. This is not a criticism, just an observation about human nature. People in a club need a shared identity.
Especially if the interviewers make comments like "um how much do you actually practice law?" if you stumble on one of their pet quiz questions.
How are these analogous? LSAT is the Law School Admission Test. It gets you into law school. It doesn't get you a job as a lawyer. (The bar exam doesn't get you a job as a lawyer either.) And you only have to take it once, not for every job interview.
Getting into law school is the first part of the "interview." If you don't have the LSAT score, you won't get into a prestigious law school, full stop. If you aren't in a prestigious school, then a huge swath of elite positions are permanently closed to you. (In general -- yes there are a sprinkling of exceptions.) The bar exam has nothing to do with it -- everyone has to pass that, no matter what you do.
The second part of the interview is law school grades and certain extra curricular signals. You're right, there is only one "interview." But the analogy still holds up, with respect to the nature of the intelligence and memory tests.
You can go to a top tier university, land a great job for several years, have positive references, and still be required to do leet code for a recruiter... that seems like ongoing gatekeeping, rather than a one-time effort/pass.
When I'm interviewing a senior developer that will guide other juniors, they will need to prove they know more than a for loop. They will need to show they understand the underlying concepts and technologies. Else it's a no hire, tough luck.
We also fired plenty of people that weren't up to the quality we expect.
Were are you working? At some government?
If they come up with, or start walking with an interesting solution, I listen to them.
Tabs and spaces don't matter.
I actively work against the biases of "whether I like the person or not" because I'm aware of that as a factor.
We can absolutely minimize all of those peripherals when we explicitly train against it and a good programming interview doesn't have to be about that.
But to be perfectly unbiased, the process would have to double blind. And I don’t know anyone who likes to hire people (or anyone who wants to be hired) following a double blind process.
So at least there is _some_ non-technical bias that gets in the process.
It’s hilarious listening to people review candidates on third coding interviews. Usually it’s “this person didn’t know this one thing I think makes me really smart”.
This is why I lean so heavy in OSS to review candidates. I can see how they interact with people, and whether they are actually able to build software that _people want_
I find with this approach it’s a lot harder to justify nonsense.
I hear people say maybe they don’t have time for OSS and then hand them a Leetcode interview, which as we all know is just about memorizing problems. OSS is such a better metric and has the added benefit of improving the world.
Yup. I just went through a round of interviews at multiple companies. In the past I successfully interviewed at a FAANG.
I agree completely.
This.
There is absolutely no need to ask such ridiculous algorithms and advanced equations questions that you will never use or implement from scratch yourself, or asking the candidate to create a formal proof by rote for a entry-level software engineering position or even a senior level position.
Such interviewers who ask these questions have both googled the answers to these questions before the interview and also have never used them practically, but only do this to appear smarter than they actually are. Worse part is not even FAANG asks any of these formal proof questions, but only some certain startups that the last time I have checked, they have failed to make money and have shutdown. Why? Because they seriously thought they were a FAANG company.
Open-source cuts through the algorithms nonsense, saves time and gets a realistic idea of what the candidate has actually worked on in the open. I can ignore the typical hello world, or demo projects but can ask for if they have created open-source libraries or contributions to any serious projects like compilers, OS projects, or any related project to the job description etc.
A take home project is also a fine metric, but asking questions about algorithms and finding out that the interviewer (or the company) doesn't even use them tells you that the interviewer doesn't know what they are looking for.
If you don't answer the 2 leetcode mediums in optimal complexity within 45 minutes then you don't pass. It's about coding skills.
Yet most companies interview as if they are inventing novel storage/processing mechanisms. Theory is important to understand which tools to leverage when, of course, but not to the extent that it's typically prioritized in the typical interview.
I've had plenty of people crush the theory portion of the interview and be poor performers on the job (slow coding, low volume output), but I've never had somebody crush the coding portion (implement very fast and proficiently), and perform poorly on the job
Even as their manager it’s not my job to lead a death march.
Somebody who produces 2x at same quality of work is worth 2x the other person, fairly objectively.
It's common to find people who produce up to 10x the average. Anyone in a startup or growth oriented company is smart to optimize for these people.
The most proficient and fast coders often write higher quality code too, in my experience.
People who over index on comp sci theory often are lacking in theory underlying good code. But it's not mutually exclusive. Just that if you have to optimize for one, coding proficiency matters much more for startup or growth type companies.
Exception might be like an AI/ML based startup with a lot of theoretical challenges. Most companies much more business/rule oriented though
A long time ago I managed a developer who was a genius at finding and fixing bugs - so I thought I would reward him by writing a new module we needed from scratch. He could not do it - when faced with a complex mess of existing code he was perfectly happy and could dive in and fix stuff, faced with a blank sheet of paper he froze and didn't know where to start.
Developers are people and all people are different - we all have our strengths and weaknesses.
It's _very_ easy to measure how fast it takes a ticket to go from "in progress" to "done" but measuring whether code has bugs, or leads to downtime or is just impossible to parse is a different problem.
If you never need code to be readable or documented I can write it quite quickly. If you want something other humans can have a hope of maintaining, that might take a bit.
Besides a 10x developer cannot be uncovered through a leetcode test or a coding test. A 10x developer pushes back on requirements, bridges gaps.. identifies issues early.. it's about making things happen not typing faster or knowing the stack well
Being able to come up with an algorithm in a reasonable time doesn’t make it crappy. No wonder you can’t pass those.
I think the reason for that observation relates to why these interview tests are generally a waste of time.
Based on the tests that I've seen, they are 'recall lots of useless information' tests. That means it's actually possible to study for these interview tests (if you have the desire) and if you can ace the test by just spending time memorizing answers.
However, 'slow coding' has very little to do with coding and everything to do with problem solving. You only learn how to problem solve through lived experience and many people never learn that skill.
These interview tests generally fail only because they're not actually testing for the skills needed to do the job.
What does that even mean?
In an interview context, I've seen cases where somebody fully understands the logic underlying the solution, yet it takes them 30m to write it, while another person can do it in 5m or less.
Essentially how long it takes to translate from mental understanding to working code
Congrats, you just invented whiteboard interview.
Leetcode-style interviews became popular in the mid 00s, primarily because they were used by hot tech companies of the time. The thing to understand is that at that time, the idea of asking people to write code during an interview what sort of revolutionary. Prior to that interviews had little structure. It wasn't unusual for the hiring manager to make the decision, some times based on credentials, recommendations, or trivial-like questions.
This type of interview became wildly popular because it allowed budding unicorns to hire programmers of high quality at scale. The process was less biased than the alternative, reproducible and scalable. Here you have two blog posts [1][2] that show the line of thought at the time.
The reality is that big tech has elevated leetcode type interview to an art. They have reached a near local optimal through years of experiments and refinements. It is working well for them so they don't have the need to take big risks such as completely revamping their hiring process.
I love the topic of hiring and interviewing and I'd love to truly get at the bottom of which method works best. I like this article because it explicitly calls out shortcomings with typical alternatives that are not usually mentioned. I hope in the future a new crop of unicorns can take these practices to the next level and do a fair comparison.
[1] https://www.joelonsoftware.com/2006/10/25/the-guerrilla-guid... [2] https://sites.google.com/site/steveyegge2/five-essential-pho...
And I think it's a great idea, personally I would never consider working anywhere that hired devs without seeing them write some code. But my problem is with the types of questions asked at many companies, specifically the types that require weeks (or months!) of prepping.
During my last job search one interview that stood out (positively) was a problem around parsing some HTTP headers (it started simple and then had layers of complexity added as I solved each one). It was honestly one of the best questions I've seen in an interview as it requires the candidate to be able to write code and solve a problem but without requiring/expecting the candidate to have prepped beforehand to learn (or brush up on) theoretical stuff far removed from the types of problems we actually solve on a daily bases (evidence of this disconnect is demonstrated by the fact people need to prep).
I had two jobs at the turn of the decade and they were conversations about topics such as “where would I go to find authoritative information on x” (google is not the answer) and approaches to troubleshooting.
Granted, I’m a script monkey (sysadmin) but some level of automation has always been part of the job.
I’ve had precisely 1 leet-code style interview (2hours for 16 problems) and I couldn’t tolerate the pressure.
The interview I had at google was much more humane than what people are attempting to cargo cult.
I see whole bunch of people complaining about Leetcode (LC) style coding questions, but I don't really see any alternative that is as good as LC. LC interviews solve all the following criteria better than other interview formats in aggregate.
- Objectivity: can you evaluate candidates as objectively as possible as opposed to subjectivity?
- Scalability: can you evaluate many candidates fast with high throughput?
- Accuracy: can you filter out bad candidates? Yes, the filter is often restrictive to the point where good candidates also get filtered out unfortunately.
- Meritocratic: can you make sure the process is assessed based on the merit of solving LC questions well as opposed to having connection to someone you know at the company or having gone to some prestigious school or other companies?
Leetcode format does good in all criteria above, or at least better than other formats below in aggregate.
1. Take home assignments.
- Objectivity: bad
- Scalability: bad. Once the question leaks, it's harder to come up with another question easily. Also doesn't scale for candidate side either. As a rule of thumb, I always decline interviews that require take home because it's useless to try hard as this is not applicable to interview at other companies. ROI is extremely low for candidates.
- Accuracy: good.
- Meritocratic: good
2. Coding questions with heavy emphasis on practicality (Stripe & Square)
I was pretty impressed with questions from Stripe and Square. They created problems that are heavily related to string, array, hashmap manipulation, searching, replacing etc using regex and such. If you are not into LC questions, you might prefer this, but I actually found some of these questions harder than dynamic programming questions I got from Google.
- Objectivity: good
- Scalability: bad. Once the question leaks, it's harder to come up with another question easily.
- Accuracy: good.
- Meritocratic: good
3. Knowledge / Trivia questions about programming language or framework
This might make sense a bit if you are specifically looking for iOS, Android or web developer but it still will need to be paired with LC questions.
- Objectivity: good
- Scalability: good
- Accuracy: bad. Candidates can memorize and recite these answers to the questions. Also too many good candidates actually do not memorize the details about programming language or framework. Also it can easily obtained from just Googling so people can cheat easily.
- Meritocratic: good
4. Pure behavioral
- Objectivity: bad
- Scalability: good
- Accuracy: bad
- Meritocratic: bad
5. Ex-coworker
- Objectivity: bad
- Scalability: bad
- Accuracy: good. If you've worked with the candidate before, you'll have a pretty good grasp of whether or not he/she can do the job well.
- Meritocratic: bad
> Meritocratic: good
I'd go with "iffy": because of the "take home" part there's no guarantee it's the candidate that did the work. This can probably be determined with a follow-up conversation during the in-person part, but without such an addition I don't see this format as "good" on this metric.
Best I can think of is take home tests but I wouldn't want to rely entirely on that, and they have their own problems too.
I have a decade of experience, working as a senior and staff engineer at megacorporations, have experience as a research scientist, am in a PhD program for computer science research, but lets just double check that I know how to use a hash table.
If it is standard at the company to give all candidates that same question, this is actually a good way to reduce bias in hiring decisions. Otherwise, you might bias toward hiring, say, people with college degrees or people who worked at large corporations, for example. And this might unintentionally bias against certain demographics that obtain college degrees at lower rates (for example).
Also, typically the point of these questions is to see if the candidate can solve a novel problem not if they can use a hash table.
People out there who can’t code interviewing for SWE roles, I can’t speak to, but there are definitely folks being asked to be fluent in code for non SWE roles who aren’t SWEs, so it makes sense everyone gets put through the leetcode grinder rubric to calibrate their level. I don’t think this is about misrepresentation, but expectation misalignment and a crude tool being necessary to evaluate human skills against a baseline.
(nearing 40, 22 years in tech, getting out asap, ymmv)
I’m a staff engineer and tech lead at a MANGA company. I code in up to 5 languages a day. I’ve written Network stacks, compression software, virtualization software, and the backbone of multiple PaaS platforms. And they were focused on whether I knew niche algorithms (apparently based on EXT3) and other stupid stuff.
I interview tons of software engineering candidates (pre pandemic it was at least one or two a week). At least half of my senior candidates fail basic phone screener questions.
These are questions that reflect something they'd be doing day to day in the role, but also are incredibly standard in our field.
The issue in most of them is that they've taken their support network for granted. The tons of developers before them who laid groundwork, and the others currently who take care of many tasks they consider junior.
Well that leaves them in a weird spot where they've not actually done a lot of the foundational work their predecessors set up for them, and they've let a lot of their skills atrophy by delegating to juniors. They'll even say as much and say "oh so and so does that, or they had it set up and it's great"
But then what am I hiring them for? Certainly they're not bringing a wealth of programming ability over, so maybe soft skills? That's hard to hedge bets on for a SWE, maybe as an product manager or something.
At my last company, I joined as their lead developer and discovered they already had another developer in the pipeline they were thinking of hiring. They thought the guy was super senior, and when I read his CV sure enough he came across that way. He emphasized on his CV that he was not only a Java expert but an expert on the internals of HotSpot itself.
Probably he'd been saying that for years, and given how many firms use the JVM it's probably a great line. Unlucky for him he got an interviewer (me) who actually was an expert in the HotSpot internals and liked to relax by reading its source code. Guess what? He knew nothing about the JVM, not even stuff you'd learn by reading the user guide. I got the sense he'd read one or two popular press articles about it 20 years ago, put it on his CV and never updated it.
So we progress to some coding. He can't code. He struggles with even the basics of starting a new program in an editor and compiling it.
The guy was demanding a huuuge salary and great perks. After I wrote up what I'd found from the coding interview he was quietly dropped from the pipeline. They'd been about to accept because they were so dazzled by his claims but weren't checking them.
The "knew it once maybe sorta" problem is common, especially with C++ where lots of people were struggling with it in the 90s and then jumped to Java/C# as soon as they came out. But they still list C++ on their CV. Ask them to write a C++ program (that does anything at all) and they'll just refuse or state point blank they'd feel really uncomfortable doing that.
Lots of people lie about their experience unfortunately. They probably don't think of it as lying. They just write down every possible thing they've ever thought or read about and exaggerate, gambling that they'll never be called on it. And, they're probably right. That's why coding interviews took off.
I did a bunch of really quick 15~20 min pre-interviews to check if the person was entering the right process before we both wasted hours of our time. Some who were supposed to have coded and deployed whole applications didn’t know how their versioning system worked. Or they reviewed PR/MRs a lot and tweaked a few lines here and there but had little practical knowledge of issues you’d regularly hit when developing, and only heard of popular tools and resources by name.
We could still had them onboard and give them time to get up to speed, but it’s harder to do when they weren’t upfront about it and you’re there trying to guess what else they’re bulshitting on their resume.
This is a hyperbolic rhetorical question but consider how much of modern work really is CRUD API gluing, and how much of that work is not rigorous engineering, but tedious implementation as per documentation and wading through hastily-thrown together legacy cruft written during previous crunch time.
How much of that work involves novel programming and memorized understanding? Most of it requires study of existing codebases, not given a blank REPL to hash out something new.
I asked “do we want to have consistency or availability for this?”, to which the interviewer said “I want both”, to which I said “umm, I’m pretty sure you can’t have both, CAP theorem and whatnot”. We spent two minutes arguing which led to me taking out my phone and pulling up the CAP Theorem Wikipedia page, and eventually the interviewer begrudgingly admitted I was right.
I surprisingly did get an offer after that interview but I found it a little annoying to be interviewed by a person with less experience than me who I have to argue with because his understanding of computer science is wrong.
When I started interviewing I started off by jumping into what I considered to be not insulting questions, but I very quickly learnt how awkward that can be if the candidate just had no clue.
Now I start with for loops and preface it with "apologies if this seems too simple" or something like that.
You would be surprised how many people can't do a simple for loop. I mean really simple.
I really hate the contrived problems that interviewers pull from HackerRank and its ilk. Even if you could glean useful information from watching someone program a knight hopping around a phone dial pad (a real problem I've gotten, which I'm highly skeptical of its utility), it comes off as pretty irritating to the interviewee. You're not solving little programming riddles on day to day work, you're solving engineering problems, and I think that these leetcode questions don't reflect that.
I think there's evidence of this too; you can study for these leetcode questions and get better at them, so what exactly are we testing? You're not testing the person's experience, you're testing how long they spent on hackerrank the night before, or how many interviews they had recently.
I think one can fairly accurately determine my level of skill from my resume, and thus I don’t think that the leetcode interviews are all that useful. I keep hearing about senior engineers who can’t code, and maybe I’m just lucky, but that really has not been my experience when interviewing, and I don’t think “spent the previous night practicing on hackerrank” really gives you any useful information.
Should it be from university CS programs? Which are supposed not to be vocational? And would discriminate against those who are not degree holders?
Should it be from internships, which are nowadays subject to the same Leetcode examinations as actual work?
Should it be from junior developer roles, which are an endangered species, as every business in desperate need for seniors, and lack the patience to train?
Maybe open source can be a sort of free apprenticeship to teach developers these good practices?
Or maybe grinding away at Leetcode, Project Euler, and arbitrary coding puzzles is really the only pedagogical solution.
It is how people pass these tests, but it’s pretty poor pedagogy.
We have no issue whatsoever hiring juniors (we hire about as many as mid-levels), but we don't have to settle for someone who doesn't have internships or projects and the openings can be closed in 2 weeks so it seems like nobody is hiring, but actually the bar has risen (and there are plenty of candidates who clear the bar).
I am confused - how is this question materially different from asking how engineers are expected to know what they will need in order to properly do their jobs? Just how are engineers expected to know programming languages, or domain-specific languages, or standard libraries, or platform apis? Is this what you are asking?
Corporations are not testing your knowledge of algorithms per se. While that knowledge is obviously important, what's more important is how dedicated you are when it comes to reaching a goal. Essentially, they are testing your will power. You know the rules. Can you achieve success and not give up in the middle of the road by skipping, say, Dynamic Programming? They could've tested your level of dedication by asking to bake some fancy cake, but throwing CS-related questions just makes much more sense.
That reasoning completely changed my perspective about algo-heavy coding interviews.
That's an interesting theory. And they pick the goal that happens to be best suited to fresh/current undergrads from Stanford who weren't working a draining job to put themselves through school, so can afford the additional person-months to achieve the gatekeeping that was set up by people like them, for people like them.
I and many other experienced software engineers have demonstrated perseverance and rising to the occasion of extremely challenging goals. But that theory suggests they're not interested in that. Maybe they're interested in the class shibboleth that was established when computers became less of a meritocracy for people with a passion for it, and more of a go-to status job for the upper middle class?
I'm not sure how to make this come through but I'm rather sympathetic. I think practical experience & can-do coder know-how count for most of the marbles. But I also think it's impressively hard to assess by, that most of what we do is just taste & path dependence on the specific techs we've seen that happened to leave good impressions on us: not science, not knowledge, not truth: most coding, most practice is almost entirely happenstance.
But again I want to empathize. Because I think it's cruel to disregard engineers that have seen so much. But I also think you're hyperbolizing overmuch & discrediting the discourse when you make your argument so sharply pointedly. Because there's some truth, absolutely, but it's also no-where near as lopsided. For example, you talk about fresh/current undergrads at Stanford. But this is a pretty regular, basic CS path that any university student should be familiar with. I only had two undergrad classes that really talked to algorithms (Algorithms and then Data structures; OS kind of somewhat), albeit there was some computational thinking already in play at that point. Understanding complexity & algorithms is understanding the basics, it really is. If you can't see time flowing, don't know the tally & impact of the work you're asking for, you're lacking knowledge to avoid a lot of bad decisions. This isn't made up fake shit taught only to the elite.
What's alluring about this system is that it has some hard facts & theory underneath it. It's not just opinion & engineering pop-culture. The answers don't change, the rules don't change, the material stays the same, and it's all rooted in being able to analyze and understand problems, rooted in comprehension. Being able to see & understand & analyze, being able to apply some basic principles: this is a pretty sure way to find people who will be able to Not Mess It Up, who are capable of looking, assessing, & navigating through scenarios computationally.
And last and perhaps most key to me: it's also material that someone with even a modest bit of aptitude can cram for. Pick up a book like "Cracking the Coding Interview" and you can semi-accurately reproduce the output of four years of undergrad in two or three months of occasional light practice & trying. Projects like Leetcode can get you the experience. You may feel bad that what you want yourself to be measured on doesn't count here, but to people trying to go get hired, it's enormously helpful to them to have well defined expectations, to have specific kinds of tests to expect & be able to prepare & study for. I see so many protests that it's not fair, that it favors only those with the luxury of time, but those views to me massively undermine how wonderfully vastly accessible it is that there's a known, well-described, well-supported subject-area we can study. The industry overflows with things to learn, study, & know, but here's something concrete & specific, which undergirds it all, which alone might not determine your coding skill, but which does indicate you at least have some raw basic intellectual capacity to go understand problems put before you & apply some sensible computational thinking to tackling them.
There's tons of things not tested for, but by having a well defined, technology-neutral set of computational thinking tests, based on actual science with objective, factual answers, rather a much wider corpus of future/current/legacy pop-engineering-of-the-day built on nothing but wishy-washy subjective opinion, I think we achieve one of the best possible wins we can against an enemy we both hate: class-based discrimination, cruel tests that reject outgroups.
There's other ways to tease out if someone is "dedicated to reaching a goal" like actually asking them questions about their life, daily routine, accomplishments etc. I think these are much better signals than "this person can implement many sorting algorithms and solve towers of hanoi or the egg drop puzzle without a google search to refresh their memory" with an N=1 sample size used to gauge how reliably they can do that. How many people if asked to do this on a job rather than an interview a year later would then go and implement it from memory without double checking they didn't forget some edge case on stackoverflow?
I know grinding leetcode is fundamentally useless because you will immediately start losing your ability to interview once you get hired. If you don't change jobs within a year you'll need to start studying all that crap again for the next interview.
An excellent question.
If the interviewer is to ask such algorithms questions looked up on leetcode, etc, then they must be prepared for when the candidate asks if they also use it themselves on the job. If the interviewer admits they don't use it, then they also admitted that they don't know what they are looking for and really are wasting the candidate's time.
If it were me, I would look at open-source contributions (no hello-world or demo projects) where that is enough proof for me to evaluate an entry-level candidate and cut through the algorithms nonsense and ask questions to the candidate based on that which will save everyone time.
That this isn't the bulk of what we do doesn't change the fact that these challenges do test computational thinking acuity. Having the ability to see & speak computationally is a good skill, one that connects our day-to-day abstract practice with actual real processes. Being able to break down problems & analyze how to tackle them shows an objective ability to assess & work through problems. I want to work with people who can be clear, who can model & explain & step through situations.
And these skills are, generally, learnable, and relatively quickly. I disagree that these abilities fade, but yes, some re-familiarizing & re-training is probably important, especially because, as you point out, the sample size is indeed often N=1, and that's pretty wild.
If you are a no-name company paying whatever your HR department defines as "market rate" and you don't get hundreds of very qualified candidates for every role then interviewing like Google might not be a great idea.
"Great" filtering for parents or people caring for ailing family or any other non-work commitment.
I guess it's not ageism at all, right?
That may sound good in theory, but in practice I've been rejected quite a few times for not delivering the correct or most optimal solution despite trying my best.
I have two very memorable no's. They were for pretty senior/advanced IC roles. Juniors would get far more slack. I usually ask an expression evaluator question.
One person, used eval(string). Brilliant, absolutely brilliant. full points for answering, but how do we make sure the string is safe to eval? Long frustrating discussion around regexes that that maybe a junior dev is responsible for maintaining. Man, just write the parser. I know you can do it. Give me something that's something that's not _crazy_ to put into prod. Give me something that won't be a bug tarpit. Sorry man, I loved your answer. I hated your response to how this can be operationalized. Clearly super smart. Maybe I should have given the green light, but the response to criticism was so bad. There are lots of people I disagree with, that I'm happy to work with. I think I made the right choice, but I wonder from time to time.
Another person I absolutely would have loved to work with, and would have learned a ton. Very into formal methods, or slightly relaxed versions of verifying reliability. My organization at the time was pretty fast and loose. I believed they would be miserable. In the interview I told them I'd give them the green light, but they'd have a huge uphill battle.
it's so hard to tell. People are adaptable, small mismatches can be papered over, but big mismatches - it's so tough to say. Relaxing and being vulnerable on both sides is so hard to do well. Maybe I was just a jerk. I think I made the right calls, but it haunts me.
Personally I don’t mind a whiteboard session or two during a loop but what’s wild to me is how, especially at big companies, you’re expected to do four or five of these to get an offer.
How often do these companies decide “well they understand when to use DFS and they can merge-sort a linked-list and they knew to use dynamic programming but we can’t hire them because they couldn’t remember how to implement a heap”?
The more casual, flexible, subjective coding interview described by the OP is not the prevalent form of whiteboarding interviews, which has been superseded by the weeder version.
I agree that this is unreasonable but not for the reason that you mention.
Once you can pass this kind of interview, you can throw me 10 more and I will still pass them.
Edit: on further thought. I think I’d be ok with coding interviews if the interviewee got to bring in their own coding question and the interviewer had to do it as well. That way the they also knows the people they are working with can live up to their ideal of a good coder.
It's saying coding interviews are not spectacularly awful if you (thoughtfully) change them.
Great! That's what everyone complaining about them has been saying in one way or another all along, change them.
I wrote about my own experiences with coding challenges during my recent job search on my blog at https://blog.urth.org/2022/04/19/software-job-search-2022-re...
I'll summarize my conclusion about live coding challenges, which is that I'm not convinced that my performance on these challenges reflected any abstract skill I have. Instead, they mostly reflected the fact that the problems I was given were either nearly identical to work I've done in the past, or were similar enough to things I'd done recently that they felt pretty easy.
I guess there's _some_ signal in that, but I don't know if it really says anything about how good I am at coding in general.
Here's some data models, and here's how you can reference eachother by properties. Cool. Let's do 3 levels of nested loops to extract the relationships and map them. That's like 99% of what they'll be doing.
The signals I'd look for are "do they start the entire thing by looking at structures or not", etc.
But unfortunately this breaks down when you're interviewing 10-year+ candidates. It doesn't yield anything useful. So sticking for those positions with a really really basic problem and pseudocoding it, and using the remaining time to instead have them map out solutions they did in the past seems to be useful.
Most things leet code interviews screen is "does this person know how to cram" because I tell you, everyone I know who aced leet code interviews crammed for 3 months and literally knew any one you can think of, and every variation. Juniors tend to do better in those, ironically enough.
Is it ironic or intentional? Seems like a great way to hide age discrimination under the guise of "fairness" (everyone gets the same test).
As I commented above, I successfully passed multiple leetcode interviews in the last four years, including FAANG. I did not spend a single minute on prep.
A) SWEs are shit behavioral interviewers in any other way. You are not about to chnage that even if you are google.
B) It is crucial that newcomers be hired by their peers and by their direct manager. This has solid research. People hate it when they get an hire and they hadn't a meaningful part in the decision... The new hire is more likely to fail.
A+B means it is better to have a terrible process lead by SWEs (e.g. coding interviews), than a good one lead by professional recruiters (which will fail, no matter their capabilities and methodology) (and yes, I do believe that without considering A, some non technical recuiters trained at workplace psychology will have a much better success rate without needing any technical interview. They will need someone technical to have a talk with the candidate, but definitely not to ask leetcode etc).
It is an interview about your communication, whether the code you write can be maintained, whether you can inspire/build trust (can you explain what you're doing? does that feel reasonable to the other person?).
Where people fail is to think that it's a test of experience, or a "heads down I must make every test case pass", that the outcome matters more than the process.
It's all about communication.
The "write code" types of interviews fit for juniors who might not know how to even write code properly. If a person doesn't know how to do that and is a junior they usually have the right energy and can put 100% into the job to adapt/learn. I hired such juniors who had no experience in the languages/platforms and were productive within a month. They had good character and discipline.
Distilling a question like that for an advanced coder is impossible. No wonder the article didn't provide any sample of a "good interview". We'd all burn him at the stake because we'd all hate it. There's no such piece of code that would provide a valuable data point about a candidate. So this is a waste of your time when dealing with a junior and with an advanced coder.
Coding interviews are difficult. You will make mistakes, this is unavoidable. I think the best mistakes to make are the ones where you hire people who are part of your team and can grow with it. Even if they aren't the best coders around, being part of the team will help them hone those skills and evolve. If I had to pick a 10/10 coder or a team player I'd pick the latter every time.
Got the job.
Then I did a slight pivot to get into BigTech via cloud app dev consulting - “application modernization” [2] - still with no coding interview although I still do the same type of enterprise/cloud app dev I did in the latter part of my career in the “real world” - and a shit ton of yaml.
Even in 2020, if I couldn’t have gotten a job at either Azure or AWS, I probably would have done what it takes and practiced to increase my income by an easy six figures or more.
Coding interviews are a “gravity problem”. You can not like either. But they still exist.
That being said, if I were starting out today and could break the can’t get a job <-> don’t have experience cycle by “grinding leetcode” for few months and and end up in the top 10% of income earners in the US right out of college, I would have definitely done it and I encourage my younger relatives graduating in CS to do it.
However, if you’re just hiring for your standard yet another senior CRUD developer and paying as such ($90K-$160K) - don’t expect me to do the monkey dance.
I think algorithm style coding interviews are the great equalizer. Even though I at 48 probably couldn’t pass one without some serious practice.
[1] https://www.hanselman.com/blog/dark-matter-developers-the-un...
[2] yes it’s a full time position with the same compensation structure.
I'm finishing up a personal open-source project and am looking for a new position focused on ruby or rails, so if you're hiring and are open to evaluating me based on something cool I've written, hit me up.
And here's where our industry isn't a "real" profession yet. There are plenty of qualifications one can get, but none that gates the ability to call oneself a software engineer. The result: we have to prove it in interviews.
And I don't know whether this is better or not. It is different though. The price of not having to sit through exams is to have to jump through hoops.
The thing with software is people tend to think of it as something that needs maintenance to stay fresh, perhaps a bit like piloting an airplane.
Airplanes have logs and pilots recertify on an ongoing basis, but software doesn't work like that. Someone might have stopped coding but still have a title.
I'm not saying software actually does work like that, but I'm sure it's a view some people subscribe to.
Obviously it’s pants on head stupid to do that if you want a web dev to make a pretty app, tie frameworks together and liaise with 3rd party integrators. However, I think many employers want the crème of such devs who are also good at algos.
Which is not going to work for the vast majority of devs and jobs.
Most devs either don’t care or need to care about algorithms, and the few devs who do, spend most days not using them.
This would be akin to asking a civil engineer to build a wooden road bridge. Having the skills is great, but how often is the company going to need them?
Maybe not in the 21st century however, and I don’t think your analogy is perfect.
It might be more like wanting to hire an architect who is also great at civil eng. The skills definitely synergize and can help prune the architect’s search space of good designs, but such a candidate is probably just going to go for an engi job, and lean on their architecture skills there: for example an algo dev that can do mock-ups.
I think that many employers don’t have a clue what makes a good dev, in general, or in the positions they’re trying to fill.
A completely fictitious language is developed. Simple enough that you can become fluent within an hour. From this you read a bunch of problems in said language and must solve them.
A set of assertions about the problems are written down and you must verify them. Finally a full program, created from a dialect of the given language is given and you try to understand
I’m also not familiar with other fields, but are doctors or lawyers asked to do anything analogous to code assessments during interviews?
Drive, experience, intelligence, communication skills, teamwork, emotional intelligence, grit, curiosity, physical health, mental stamina, family/home situation, and dozens of other factors play into job performance, many of which you can’t even ask about to generate your test dataset. Many of these traits are what you’re trying to measure the effects of in the interview.
a) Its like saying a Tobacco company has a paper saying smoking is healthy
b) Google measures things that matter to them. The operate on a scale that 99% of companies aren't. Their predictor of success only applies to companies of their scale.
c) What is their definition of success? And how does it apply to smaller companies less than 1000 employees?
What works for Google does not mean it will also work for the 99% of companies that are NOT Google or any other FAANMG company.
This only shows that we have an incessant coding interview cargo-cult around how FAANMG companies interview candidates and several startups taking that to the extreme and thinking they are working on complex Google scale problems.
To these interviewers at these startups still under this delusion and for the 900th time as many have previously said for years as a wake up call:
You are NOT Google.
https://news.ycombinator.com/item?id=31544634
How do you test knowledge indeed?
Lol be careful not to ask for a loop, nobody uses loops in real programming problems
dont mind talking over code casually but dont put me on the spot or give me homework. not doing that.
I just want to put this out there to anyone who gets involved in interviews that this is a very important rule -- if your question can basically be redefined as some really clever solution you found on a problem you put > 30 minutes into, it's probably not a good question.
A lot of the seniors I work with would come up with questions like this, and I get their logic:
1. I'm a senior and this was a challenge for me 2. The logic is clear when you see it and look at it 3. I want people who can do work similar to me 4. I know how to get to the conclusion, so I can judge based on the path the candidate takes Conclusion: It's a good question
While I understand how we can get to such a conclusion, I think that in most cases the only takeaway you can get is "can this person think like I did in this situation?", which maybe has some value for judging workplace compatibility, but I don't think it's a fair assessment of someone's technical aptitude. If you yourself required a lot of time and brainpower to sort through a tricky issue like this with the benefit of the problem scope and system needs that led to this issue, it's very unlikely you'll be able to judge any assessment
Point 4 seems good at first because we can say "the goal is to just see them think and use their experience"; there is truth to it but without the rest of the context it's not really a good assessment because ultimately, you're looking to see "did you get the same conclusion I did? Did you avoid the same pitfalls I hit?"
Instead, focus on a general scenario that tests their understanding of basic principles. How do they use the fundamental knowledge for the subject matter to work through a new problem that you yourself haven't really dealt with. It removes your bias a bit and opens you up to those "wow why didn't I think of that?" moments that you'd have working on tough projects together.
Try your best to give a good faith interpretation to any path the candidate takes and help take it to natural conclusions, and see how the candidate reacts; if you can see they've made a workable but problematic solution, nudge them towards the problems you see and see how they discuss it.
For me a candidate that can really process their own thoughts and do a good analysis of their own solution (its benefits and shortcomings) is extremely valuable; I usually lead off with "I know how I'd do it, but that doesn't mean it's the right answer, so focus on your strategy and walk me through it." If you want tot test compatibility or you feel you have a stronger solution, accept their solution (if it's valid) then go ahead and discuss your thoughts a bit and see how they respond and how they discuss it. The more you make it a technical conversation against peers and less a university exam, the more you have to see how they think and operate and the better you can understand them.
It amazes me how many people will defend the current interviewing process and then openly admit that the majority of their engineers could not pass the same process without spending dozens of hours preparing.
[1] https://news.ncsu.edu/2020/07/tech-job-interviews-anxiety/
But other than that the study didn't find any evidence that men have any issues with being observed while coding. So unless a better study comes around we can dismiss complaints about observed interviews being stressful unless it comes from a woman, it is in the study you linked after all. (I do believe there is a stress effect, but that study isn't showing it, need a bit more samples and probably several different observers to get better data)