What's really wrong with these developers: undertrained, low mental horsepower, bad "fit", imperfect industry experience? All of these are true, but 99/100!?!
With such a developer shortage, can these companies really not get productive work of a slightly broader group?
Something isn't fitting together here.
Relative to the 1/100 resumes-- if your friend took the time to dig deeper, he'd likely spot 2-3 diamonds in the rough, a moneyball misfit.
If your friend is like most senior managers, he'll scoff at the idea-- 'I'm too busy to read 100 resumes' he'll likely say, or 'that's HRs job!'
Finding & attracting talent is a mission critical competency that senior leadership must drive. Too often it's delegated to mid-tier management that don't share the same sense of urgency.
It takes an exceptionally rare manager who can assess and train-up mediocre talent into A-Player status. Fewer still, who have the guts to make a trial hire. A bad hire can be costly on a variety of levels.
But most guys would rather play it safe-- blame HR, lack of bandwidth, or the market. Meanwhile, the guys who figure it out manage to scale.
Finding talent isn't the only challenge. Incentivizing them to join and stay is equally challenging.
From what I've seen, the other 99% of resumes are people trying to break into the field but with no actual programming experience. Competent developers with a track record get snapped up really quickly, while newbies without a track record go on to apply to 100 more jobs.
You can also blame it on an unrefined interview process. I've probably turned down the right candidates accidentally while trying to work out an interviewing system. With a startup, its hard to get enough feedback because you can't hire enough people to verify if your assumptions were correct.
Another issue to consider is unrealistic expectations for a candidate (has to know X, Y, Z frameworks + machine learning + mobile) If this candidate exists, you probably can't afford them.
For the rest of us, I strongly believe you can train most developers to do the job. It's how much time you're willing to invest in them. I've taken inexperienced developers that are very interested in the company vision and given them a chance to grow in the roles. After a year, they're the "good" developers I've always wanted.
Invest more in getting the right kinds of candidates.
Thing is, companies and jobs differ, and so do people. I've seen top performers from company A fail in the interviews at company B, and vice versa. Leaving both companies under the impression that the other company employs 'bad' developers.
Whether or not your skills+potential can be used optimally by a company depends on so much details, like cultural fit, to which extend you agree on their paradigms (methodology, problem solving ideas, prioritization, etc).
Long story short; companies are usually looking for the most optimal choice for their job openings, and so are applicants. This results a way more complex matching strategy that just 'could do the job'.
Perhaps John Nash can help us with optimizing these strategies. Until than, I think 1:50 to 1:99 ratio's are here to stay.
I screen a lot of ML researchers and engineers.
The process typically starts with a 30-90 minute phone screen (it ends faster if I don't like you). I start by discussing the company and the position, in as much depth as the candidate wants. Then we discuss one project of the candidate's in depth, both for some technical expertise but also to suss out what the candidate, as opposed to a team, actually did. Good candidates should be able to discuss what the project did, why, implementation, challenges, problems overcome, tools used, etc. Then we do a quick survey of general ML technologies; we're hiring senior people so they should have a pretty good breadth of experiences. So they should know their way around regression, trees, forests, boosting, svm, linear algebra techniques, plus optimization tricks for all of the above. It's ok if a candidate hasn't used any one specific thing, but they should have a rough knowledge of which tool to use when, plusses and minuses of tools, and the typical ways one improves results (hold-out testing, k-folds, l1 and l2 penalties, etc etc.) If you've actually used most of these tools you should easily pass the phone screen.
Then we bring people in or fly them out. Here, we go more into depth in several tools. You should know the math and the practical use of several ml techniques really well. We'll also make sure you can write enough code that you don't have to be handheld. That is, you need to be able to do some data processing and string together data pipelines to get your work done.
common resume failings include (NB: this is for an ML engineer / data scientist):
* (very common) no academic prep and no demonstrable experience. ie a potential junior person applying for a senior position. If he or she has no degree but spent 4 years at netflix or somewhere good, we'll definitely at minimum talk on the phone. Demonstrable experience definitely means education barely matters.
* you live abroad, thus requiring sponsorship and a move, and aren't a really strong candidate
* (surprisingly common) you spammed all our different jobs, making us believe you just want a job, any job, and paid no attention to what we're looking for
Common failings from a phone screen or interview include:
* (weird, but happens) don't make the agreed upon phone screen time without an email before (or even an email after, if an emergency occurred)
* (really weird, but happens) you have no idea what the company does. I'm looking for you to have spent at least 5 minutes reading our site and have a 1 sentence summary of what the company does. If you haven't done even that much research, why on earth is the candidate applying to us? A perfectly fine example would be, for example, if I worked at google: You build a search engine and you want ML people to improve search results. At, say, square, a great response would be "you're handling monetary transactions and therefore have fraud problems from card holders and probably want to build risk management tools to protect yourself from bad merchants." I find it really weird many people don't even know that much. This isn't an instant fail, but it's a negative sign.
* he or she wasted his or her phd. They may (or may not) have a very deep understanding of one tiny piece, but lack a decent understanding of the breadth of the field. We need breadth. And yes, they may be able to learn, but the question is then what the hell did they spend a 6 year phd learning? Also, we're hiring senior people. And again, anyone with a good ms in ml should do fine (not just a theory; we've hired them from stanford and ga tech).
* They don't know deeply at least one or two techniques and tools. Whether its hadoop, vowpal wabbit, R, sklearn, weka, whatever, you should have become an expert in at least one tool and know it inside and out.
* (very common) an inability to program enough. Obviously most ml people aren't engineers, but they have to be able to do data processing, data cleaning, run ml tools, build their own ml tools if necessary, etc. Senior people in yahoo or google's research labs may not need to program, but for us you do.
* (somewhat common) bad communication skills. We speak english; you need to be able to speak reasonably clear english so that we can discuss technical topics clearly. How people can finish a long stretch in an american university without this escapes me. And to be clear, about 1/2 the employees here are foreign born, so we're pretty good with different accents and the typical indian and chinese esl idiosyncrasies.
The last thing I can say about job seekers is we're talking because I have a problem. That is, I really want you to succeed if there's a good chance this will work.
1. Hire depending on your team. There are candidates who are amazing data scientists but do retarded shit like select * from table order by rand() on a ten billion row table and bring down the cluster. Sometimes, it is OK because your team has incredibly strong engineers who can work in close combinations with these people. Sometimes you can't really afford that. At my current workplace, my manager did a really good job of hiring people who are fullstack (data + backend + viz + ml) while having strong preferences in one direction or the other. So you might have an incredibly strong ML engineer who can roll his own ML code out on Spark, while also being open to writing back-end infrastructure code once in a while. Or you might have this guy who thinks in data pipelines all day long, but will spend a bit of time thinking about mathematical model. The teams work well.
2. Hiring for ML teams is really really hard. When I was in San Francisco at my previous job, we used to screen quite a few candidates everyweek. We would see failures similar to what you posted. Either engineers who knew fuck all about ML or people who were good at the theory but didn't know how to code. Somehow, at my current job in NYC, we lucked out because we have a bunch of really awesome people who moved out of finance but want to continue to live in NYC.
> They may (or may not) have a very deep understanding of one tiny piece, but lack a decent understanding of the breadth of the field.
This is close to what I have seen from both sides of the table. But, one thing that annoys me is the number of teams where people have pet algorithms that they are obsessed about. They keep asking questions about those pet algorithms and ding a candidate for not knowing that specific algorithm in depth. One should be able to be flexible while doing this.
> They don't know deeply at least one or two techniques and tools.
I think we see people from so many different walks of life that for junior people the goal is to make sure that they know the shit that they have in their resume and drill them if they don't. As in, we don't hire for specific techniques or tools because the team has its own collection of tools/libraries (Spark/Hadoop/Python/Word2Vec/CF etc) which you will be able to pick up fast if you are good enough.
> I think we see people from so many different walks of life that for junior people the goal is to make sure that they know the shit that they have in their resume and drill them if they don't. As in, we don't hire for specific techniques or tools because the team has its own collection of tools/libraries (Spark/Hadoop/Python/Word2Vec/CF etc) which you will be able to pick up fast if you are good enough.
I think you misread what eshvk was saying; he isn't saying that they don't know one or two specific techniques or tools, but rather they don't have any tools/techniques at all that they know deeply. If you read what followed the part you quoted, it should become clear.
* As far as graduating seniors, when you post a job offer, you get maybe 10% of the resumes from people with either a CS/CE major or minor. That's not to say they can't code (in my experience a resume from a non-CS major with good experience is a much better bet than a resume from a CS major with little or poor experience). But we literally get people whose experience totals up to "took a 'How to write HTML' class once freshman year as an elective" applying to jobs involving low-level C
* Actual on-site interviews are very expensive; unless you are really hurting for people, you need a cheaper (and potentially less precise) way of weeding things out.
* With the exception of graduating seniors, there really are relatively few good candidates actively seeking work; most of the good ones have jobs (and you can hire them, but that usually isn't counted in the X% of resumes get offers, which usually refers to responses to a job posting; hiring people away from their current jobs usually starts in a different manner).
* On the other hand huge numbers of people who are desperate for work send in resumes. The less harmful of these are the ones who obviously are unqualified and clearly just spammed every job offer listed regardless of the requirements. Since we aren't talking about graduating seniors (see above) the majority of the not-obviously-unqualified often even have relevant work experience, but that's just because it takes time for people to fire you.
I am talking about people with 2-5 years of experience working in language X who are barely able to (or sometimes even completely unable to) write FizzBuzz in language X.
Another example was someone with over 5 years of OS kernel development, who in the interview was unable to describe what exactly they did in those 5 years and they didn't seem to know a much about any of the various OS topics we tried asking them about: scheduling, interrupts, DMA, filesystems, memory paging, IPC.
These people are particularly harmful, since there are very few tells at all on their resumes (and no reliable ones), you would have to interview all of them, which gets expensive. The majority can't get past a phone-interview, but when you are talking about
This leads to a lot of companies not even bothering to interview people (again ignoring those graduating from school) without referrals from current employees, or some other way of weeding out the massive numbers of "good resume, but bad candidate" submission that also will, as a side effect, weed out some of the "good resume, good candidate". It also means that if you are looking for work, and can't find it, then either you are unqualified, or need to get referrals from friends who have jobs at companies that are hiring.
