Whenever you evaluate a candidate on any axis, you have to be very careful about whether or not you’re judging them on something related to work performance or not. In the case of GitHub profiles, you’re also evaluating someone based on whether or not they have the post-work time to contribute to open source work, which is unrelated to their ability to contribute during working hours.
I’ve hired enough amazing engineers who didn’t have time for open source contributions that I’ve decided that open source work is not a meaningful metric for evaluating a potential hire.
Also, my GitHub profile is not impressive; it would be deeply hypocritical for me to hold candidates to a standard I couldn’t meet.
I won’t try to guess how good they are at coding from GitHub alone. Most of my own public contributions are cases of “four days studying, two hours coding.” So as an exercise in tempering my interviewer ego, I remind myself that by just looking their work, I am very likely missing a lot of the picture.
Amongst the many many ways code evaluations fail, the worst and most typical IMO is that the evaluator has an air of superiority, who marks down things they don't understand, and thinks their own coding is that of an artistic genius, approaches the tasks with zero science or rigor and is unable to articulate anything hard to back up their vague assertions coming out of the assessment "process".
Besides normal open source contributions and semi-maintained side projects my own github for example is full of one off scripts and weekend project repos that are interesting for an average of N<10 people per year, I still put those up because that N is larger than 0. You won't find the code quality your previous post asked for, or even any test code in some of them. That doesn't mean I don't have any. Just like there's great developers with zero presence on GitHub out there. It's just one of the many signals in the hiring process, as long as I don't see a major red flag in the profile it mostly informs "this person works on interesting stuff in domain X / similar interests like my other devs / extensive experience with platform Y". I don't have time to look at fancy coverage badges in all 30+ repos of a profile, there's plenty of time to talk about specifics later on. I treat github profile links like any other, say to a candidate's personal tech blog. The mere existence is nice and I'll give it a glance but that's about it.
In most domains I've worked in other parts of the CV and public presence are far more informative about a candidate, shoehorning pseudo-scientific metrics about code or test quality on candidate's github profiles seems counterproductive. People have other things to do in their life than maintaining recruitment-friendly github profiles, expecting one is not much different to the odd practice of unpaid take home exercises imho.
If someone doesn’t work out we just let them go after half a year. So far that hasn’t actually been necessary.
Does the distinction matter at your company, or is the primary metric "good enough"? Is this industry standard, or does it vary by company?
I definitely don’t rely on their GitHub info for that. In the interview process in general, I don’t expect this with junior developers, only senior+ developers, and then I look for how deliberate the developer seems to be with design trade-offs in their coding exercises and their answers to “tell be about a time when...” questions.
Does the distinction matter at your company, or is the primary metric "good enough"?
The distinction matters, but in my experience interviews produce very noisy signals for this trait. (1) How often do you design API’s in half an hour? In practice the decisions that affect long term maintainability happen over days, not hours. (2) Evaluating this requires a lot of subjective judgement from interviewers, and experience tells me that no interviewer is as objective or insightful as they think, myself included. So instead of trying to answer that question, I just look for a deliberate process of enumerating and deciding trade-offs, with answers that are in the ballpark of sane.
I wouldn’t call good enough a metric, more like a philosophy for hiring. I look at many unrelated traits in an interview, but I explicitly reject the idea that the interview is accurate enough to conclude something like “this developer is a net reducer of technical debt”.
Is this industry standard, or does it vary by company?
The interview rubrics I’ve seen all use holistic definitions of “good enough to hire”, if that’s what you’re asking, but based on what I’ve seen as an interviewee I highly doubt there’s an industry standard the way you’re asking.
