The problem isn't the tools - they're inevitable. The problem is that our industry clings to this bizarre ritual where we test for skills that are completely orthogonal to the actual job.
My current team scrapped the algorithmic questions entirely. We now do pair programming on a small feature in our actual codebase, with full access to Google/docs/AI. The only restriction is we watch how they work. This approach has dramatically improved our hit rate on good hires.
What I care about is: Can they reason through a problem? Do they ask good clarifying questions? Can they navigate unfamiliar code? Do they know when to use tools vs when to think?
These "invisible AI" tools aren't destroying technical interviews - they're just exposing how broken they already were.
Whenever the topic about how broken tech interviews are has showed up on HN in the past, there were usually two crowds: the "they suck, but they're the best we've got" people, and the much less common "they suck, so we do something else" crowd. Almost everyone agreed they were broken.
What does it say about the tech industry that so many orgs continued to use a system that was known to be broken for so long? How much inefficiency and waste over the past couple of decades is attributable to bad hires? And conversely, how many efficiency improvements in the near future are going to get attributed to AI tech rather than the side effect of improved interviewing practices meant to combat AI candidates?
Well, broken in what sense? In the sense that HN complains about it? Then it says nothing at all because that is not a signal of anything. HN reliably complains about ALL interview methods including coding challenges, chats about experience, take home exams, paid day-of-work, and so on.
If it is "broken" in another sense, we would have to quantify that and compare it to some alternative. I'm not aware of any good quantitative data on this sort of thing and I can imagine that the challenges to meaningfully measuring this are considerable. A qualitative survey of the tech industry is likely to surface opinions as wide ranging as HN complaints.
Basically you start with the conclusion that what we have is extremely sub optimal, but that's not clear to me. I think it is uncontroversial to assert that coding challenges are not an amazing predictor of job performance and lead to lots of false positives and negatives. Easy to find lots of agreement there. What is unclear is that there are other methods that are better at predicting job performance.
It may just be that predicting job performance is extremely difficult and that even a relatively good method for doing so is not very good.
That they tried to scale hiring. I don't think that technical interviews are bad per se, it's the form in which they are currently that is bad. I was once applying for a company working on a stage animation software - somebody there created a task relevant to their workflow, they specified which libraries I should use and then gave me 14 days, after which I sent them a link to Github repo with the code. It was overall a pleasant experience - I learned something new programing this task, they were able to access my code style and programing abilities - but this was long before AI. They were a fairly small company though.
In the large orgs you have a HR person overseeing a hiring who doesn't know how to code (fair enough - not their job) and bunch of engineers usually far to busy with actual problems to create a problem set and far to many applicants. So companies drop leetcode at the problem - thinking that any selection criteria is better than non selection criteria.
I'm in this camp and I don't think it's "broken" - at least not in the sense that we have a broadly applicable "fix" that improves the situation.
I.e., everybody hates coding interviews but it's still close to the best we have got, and it beats credentialism or talking about random stuff (how are you going to compare across candidates).
Many? Yes. Almost everyone? No.
I agree. And it's not a good sign for the industry when engineers have to spend months working up to being good at answering these questions just to get the job.
You’re spot on with the uselessness of modern algorithmic questions. We live in a world of high level languages; in 20+ years, I’ve never implemented a single one of those coding problems in an actual codebase. I would prefer to hire an intelligent, sociable person with core skills and who has room to learn and grow, rather than someone who can ace silly algorithms but can’t work with a team or complex codebase.
Heck, the person you refer to having hired, and presumably fired, may not have even used these tools. The trend these days is to learn to the coding tests from sites we all know about, not the ideas or concepts needed to work in a real team on a real application. They spend all their time writing algorithms, but often never actually touch a database/write a real complex application. The bootcamp explosion was a self-escalating problem for algorithm-based test questions.
I've been on this gig for about 20 years, this process is by far the best one to participate in an interview both as an interviewee as an interviewer.
As an interviewer I have a lot more joy to tackle an actual task, or a problem very similar to how my team works, with someone applying for the position, it's easier for me to feel like a human instead of an assessment machine. I don't have to learn the N different potential case studies to run the interview, I don't need to feel like I'm ticking boxes when giving my feedback, it's all natural.
As a candidate it relaxes me a fucking lot, I don't have to brush off old books, go through a grind of studying and still feeling like I haven't studied enough because there's an endless amount of knowledge to learn if I need to cover all bases. I also feel a lot more like a human, I can talk through my train of thought, work as I normally would checking documentation, searching, etc.
This bizarre ritual feels almost like hazing by this point, at some point Big Tech decided this was "the way for hiring", then herd mentality took over to the point where tiny startups just cargo-culted processes without even questioning (I heard from a non-technical founder once "if Google does it then it's the best way to do it"). A few generations later of folks hired through this Byzantine process turned it into a hazing ritual, if they had to go through this pain then they might as well inflict the pain to make candidates prove themselves worthy.
It's a giant cycle of bullshit, and no matter how much I tried it's been always impossible to change the minds of HR that this is not the best way to assess candidates, the herd mentality is even stronger around there since there's always the dumb scapegoat: "we do benchmark peers in the industry and they all do this way, so we will do this way".
Sorry for the rant but it's one aspect of this job that really grinds my gears, I fucking hate running interviews with candidates because of it, I never feel I could properly measure someone's real abilities, and I really hate having to go through it myself...
Throughout the conversation, we mostly stay light and touch on a lot of different topics. But every so often, I’ll drill in and start discussing some random topic at depth. If you drill in just 2-3 times throughout the interview, you get a pretty clear picture of the candidates average depth of knowledge.
Not only is this LLM proof, but you also get a sense of their opinions, their interests, their passion, etc.
For example I was always a great employee but early in my career I wasn’t big on unit testing.
Or I interviewed for an ML job and they dinged me for not knowing a bunch of statistics things off the top of my head.
Unit testing does have drawbacks though, so as long as you could explain why you aren't a huge fan of it, I don't see why this would be a disqualification
I'm no lawyer, so I'm not sure this would rise to the level of actual legal fraud, but moral fraud at the very least is what I think I'm seeing.
EDIT:
Now I see it at the bottom of the page... "Interview Coder is a desktop app designed to help job seekers ace technical interviews by providing real-time assistance with coding questions."
So yes, it's exactly what it looks like.
The important thing is that such judgement doesn't depend on the integrity of the other party: the act stands on its own, independent of the intentions of the other party.
If you want a job that someone is offering and they ask you for irrational, unreasonable, or just stupid demonstrations as a prerequisite for getting hired, you have one choice: decide for yourself the show isn't worth the price of admission and walk away... or you do the irrational, unreasonable, or just stupid thing asked to the best of your ability and keep your hat in the ring for the gig. Either way honesty and ethics dictate that you either play the game or walk away.
The moment you cheat and lie: that's entirely on you and perhaps your own dumbass decision to train and enter an industry that works this way. Of course, I mean "you" in the abstract, not necessarily you personally.
So when an employer sees many employees and candidates work the system like this, then they are also right to say, "pretty much every employee we've hired or candidate we've spoken with tries to dishonestly game the system and deals with us in bad faith, so we're justified in screwing with them any way we please. No moral fraud here: we're just doing onto those as those would do onto us."
Well, great... you've defined a level playing field that's working as optimally as it can and without any moral blame at all.
It's not fraud.
The candidate could also use this tool to help with the job once they've been hired, and that would not be fraud either.
They wouldn't, but that's just because the interview is stupid and not like real work.
2. No, you're not allowed to lie until blue in the face when selling; https://en.wikipedia.org/wiki/False_advertising is illegal. (Is it underunforced? Probably. Do I wish those laws had much stronger restrictions and harsher penalties? Yep. But is it illegal to lie to sell things? Still yes.)
(IANAL and this isn't legal advice. Though it is moral advice.)
I agree that lying and fraud by businesses, employees, buyers, and sellers are all reprehensible. Nonetheless, I would contend there's no problem discussing the actual subject at hand without expanding it.
I'm not being hypocritical, I'm simply being topical.
As a software development manager, I find the most important quality I need in my direct reports is honesty. If you are not honest with me it makes it very difficult to do my job.
That some developers have been conditioned to dishonesty is a shame on our industry.
While I agree with that sentence broadly, tools like this undermine the process even for non-orthogonal skills. For instance, we administer System Design interviews and Practical Coding interviews (usually, we give the candidate a code base and ask them to make a modification to it)—things that are not LeetCode and are pretty relevant to day-to-day work. We actually let candidates use AI, as long as they show how they're using it. Tools like still undermine our process even for those types of interviews.
I'm a realist and understand that tools like this are inevitable. But I don't think they're ethical, and I think the "Fuck Leetcode" argument justifies their existence. In general, trickery is wrong (whether it's companies doing it, or candidates).
System Design interviews can be crammed too.
I'd find it much better if people would ask you "tell me about some complicated systems you helped build and what high-level challenged you encountered". Instead you get "how would you design google docs?".
Does “100% of their loops” mean everything after the initial contact?
I suspect smaller companies could find this challenging.
I’m old enough to remember when independent recruiters acted almost like “talent agents,” with large payouts.
This encouraged them to curate a “personal brand,” and they would often do a lot of the vetting, themselves.
I believe that executive recruiters still operate that way, but engineering recruiters seem to have sharply declined, since those days.
For smaller companies in particular the above can be catastrophic, so I recommend they adapt to changing times.
But when it came to large complex codebase or problems that required critical thinking everything fell apart.
The service on offer here is different. It's providing a means to use LLMs to fake your way through a technical interview.
Showing that you can use LLMs to quickly and correctly solve problems is a good skill to have. Offering up a solution from an LLM as your own work without acknowledging how you got there is just misrepresentation... or to put it another way is just lying. Maybe fake your degrees and experience while you're at it, right?
At least in the long run, many that need these tools to get in will be found out once they start having to solve real problems on the job. Just a shame about other, more qualified people being turned away. Of course if the LLM was sufficient enough on its own, perhaps a real software developer was never required to begin with.
The most ironic thing is that this qualifies for "hacking a system to your advantage" for Y Combinator.
Those that are upset by Interview Coder as 'cheating' are themselves bounded by an outdated system waiting to be disrupted and Interview Coder is the result of that.
The only way to stop this and is to return to onsite interviews. Which reduces cheaters to >98%.
This spells the definitive end of Leetcode and the rest of the online assessment tools.
What will happen is that they will look for cheatproof solutions, such as testing on site, and filtering by credentials. Is that really what candidates want?
And while yes they should also test for stack related and other technical aspects and not just Algorithmic leetcode esque knowledge , thats more of a reflection on the test.
Great idea! That is even better.
We have interviews where we aren't allowing the use of it (yet interviewees are using stealth AIs to cheat). At the same time, I am also hearing of organizations mandating the use of it, ie: "20% of the code committed needs to be generated". There's probably a set of orgs that exist that do not allow the use of AI in coding interviews, yet practically mandate the use of AI in day-to-day work!
We are at an inflection point I think, but my guess is AI is going to win out soon enough.
It’s a platform where referrals can register and then they put some money on the line. Say $20.
When an employer calls my referrals through the platform if I end up getting fired in the first six month the people that referred me lose their money (and reputation). If I stay on they get paid that amount of money.
Feel free to tweak the idea but I think it would be great to hire based on referrals in a trustable way.
Even before AI you would have candidates of varying skill level so your coding questions should have always scaled depending on the skill of the candidate.
The purpose is not to check if you've memorized some algorithms - it's to verify that you're capable of mentally constructing the model of a problem in your head, thinking through it in a structured way, etc.
Giving a candidate access to AI doesn't eliminate the need to do that.
Instead, our process starts with a one-hour technical conversation. We talk through the candidate's experience, how they think about systems and products, and dig into technical topics relevant to our stack (Ruby on Rails). This includes things like API design, ActiveRecord, SQL, caching, and security.
If that goes well, the next step is a collaborative pull request review. We have a test project with a few PRs, and we walk through one with the candidate. We give them context on the project, then ask for their feedback. We're looking for how they communicate, whether they spot design issues (like overly long parameter lists or complex functions), and how they reason about potential bugs.
This has worked really well for us. We've built a strong, pragmatic engineering team. Unfortunately though, none of us now remember how to invert a binary tree without Googling it..
If you have job requirements that extend beyond “trivially completable with AI” ask questions that aren’t trivially completable with AI.
The Leader of the LeetCode Rebellion: An Interview with Roy Lee (70 points, 9 days ago, 44 comments) https://news.ycombinator.com/item?id=43497848
I got kicked out of Columbia for taking a stand against LeetCode interviews (20 points, 9 days ago, 18 comments) https://news.ycombinator.com/item?id=43497652
If they seem to be reading intently, that’s a flag. If their answers are fluffy and vague and then get very specific, that’s a flag.
Tools like this might not show up on shared screens, but people who use them behave unnaturally. It’s pretty obvious if you know what to look for.
I’ve been doing dozens of technical interviews per month and it’s pretty clear when the person is Googling answers or using some ai tool.
For our business we don't use Leetcode, the future looks something like paid bounties and in person interviews.
tinycorp from George Hotz does the very same thing of paid bounties to get hired there.
The highly talented people will do this for fun, while those who aren't will self select themselves out.
No?
People who have time AND enjoy doing this sort of thing in their free time will do this for fun. That’s the self-selection, then from this pool talent hopefully gets translated into results.
Why reduce your pool to “free time and enjoy doing bounties in free time”? That’s excluding many talented people. I’ll also point out that it’s discriminatory: single childless wealthy men tend to have a whole lot more free time (for example women do most of the unpaid care work in all countries, leaving a whole lot less time for this sort of thing).
I also have a suspicion (not based on any data) that people who enjoy doing bounties in their free time certainly tend to be technically talented, but also tend to have non-technical weaknesses around communication and other soft skills. So you’d self select for this weakness too.
This is just, like, your opinion. Your future employer may think otherwise, and look for people with algorithmic skills. "But leetcode is actually evil" is just your rationalisation of your cheating.
But then of course, since FAANG did it, everyone else jumped on the leetcode bandwagon and started asking ridiculous DSA-exam-type questions that had nothing to do with their actual work, even if they had the capacity to conduct proper interviews for their candidate volumes.
I am an employer and would much rather have in person interviews than leetcode.
It doesn't test for anything that AI can do already if not faster otherwise.
Its almost useless as a way to learn how to be a better coder, as most of the "fastest" answers are unreadable.
But if you are using it as a basis for interviews, you are more likely to bump into someone who has trained on that particular question.
I'm not sure what the answer is, as other said, pair programming is kinda the answer. Maybe debugging something in your code base.
No, also mine.
If you want to filter out these candidates, bounties are the way here and in person interviews.
Leetcode can't help you here.
Wild
They could get a sense of what type of work they’d be doing and the competence of the organization and you’d get to see how they perform in the real world.
Plus the code base will involve technologies they are not familiar with; rarely does everyone know every technology needed beforehand.
The interviewee could suggest the specific implementations within that.
Eg. We need to hook into our fubu system. Here’s our library for that. How would you code that?
The engineers I’ve worked with who’ve caused the most damage by far, have have been technical tornadoes who did fine in interviews.
I’ve never seen any damage caused by someone who slipped through the cracks without being able to code at all (and I’ve worked with people like that).
I think we’d be much better off if we just fired people who outright lie about their ability to code and spent more time digging into previous employment history, talking though projects, and talking to old coworkers.
This crap is transparent. A person actually solving the problem won't type that fast or not think silently about it. And even better, the LLMs usually get stuck in a snag on some of the problems, or they write weird snippets
(though this might bring down the number of interviewers asking for leetcode)
I've found that applying MECE principles (Mutually Exclusive, Collectively Exhaustive) to knowledge domains dramatically improves AI performance in complex tasks. It ensures comprehensive coverage without redundancy, allowing the AI to navigate concepts more effectively. This seems particularly relevant for assessing candidate depth versus breadth.
