calabin

Yesterday, we did two basic-screen technical interviews where both candidates appeared to use LLMs to generate nearly their entire answers.

We do this quick screen after a 30-min behavioral interview to make sure that candidates can generally operate at the skill level they claim on their resumes.

In the past, we've been shocked by the number of people who will talk a big game, but have really rudimentary programming skills when the rubber meets the road.

The questions are:

1. FizzBuzz

2. Generate the first 20 rows of Pascal's Triangle

3. Drop all non-prime integers from a pre-defined set of 2-to-N

The first candidate we didn't totally suspect, until the second candidate provided nearly letter-for-letter the same answers (same variable names, function names, etc.).

After the interviews, we popped our deck into ChatGPT + Claude and it output exactly what these two candidates had provided.

Last week, a third candidate sent us clearly ChatGPT'd code as an example of some of his work.

I'm unsure what to do here, so I come to you HN to ask, what you have done to guard against the use of LLMs in remote technical interviews? Thanks!

Bonus: The nail in the coffin was when the second candidate immediately clocked the last question as leveraging the Sieve of Eratosthenes. Previously, he'd shown us a pretty impressive portfolio. When asked how he knew the Sieve of Eratosthenes off the top of his head, he claimed he had used it in one of his commercial portfolio projects but couldn't explain how.