Recently, I've found that the candidates in my company's hiring pipeline are using ChatGPT to solve the coding test we use for screening.
While the coding test isn't necessarily complex, it is still hard to finish it within the time limit I provide. This means that candidates who are genuinely attempting it often submit the code with some tests failing. However, some candidates submit the tests with all tests passing, which would be pretty tricky (but not impossible) in the allocated test time. Sometimes, there's a tell, like their submission time being too fast to be reasonable, but it is not always possible to tell.
I am using a code-screening test platform (CodeSubmit). Despite having challenging problems in its library, it is not ChatGPT-proof. But I'm also amazed at how well ChatGPT solves the problems (well, most of them). It seems to be able to follow complex instructions and adjust the program to pass unit tests.
Are you seeing the same problems in your hiring pipeline?
How can I ChatGPT-proof my screening tests and take-home assignments?
