Here is a blog post on my transition from theoretical physics to data science (and how it made my life much better): http://p.migdal.pl/2015/12/14/sci-to-data-sci.html
I understand that Andriej Karpathy (my favourite author/lecturer in Deep Learning, by a large margin) had a wonderful PhD, in a fast-growing field, with a golden fall-back option. But most PhD students I know (including my former self) do things in disciplines no-one else cares about and are tied to their institute/advisor/place with little to no opportunity to change things when they go awry (cf. it's super easy to change a company). A non-trivial fraction of my friends suffered from depression or had a serious mental breakdown (again, including myself).
In this light, while it contain a large number of helpful tips and valuable pieces of advise, why is it called "survival guide"?
But he's probably my favorite blogger too and its at the very least interesting to hear his take on his experiences.
By far the largest fraction of the post is concerned with tips/tricks for effectively navigating the PhD experience once you commit to going through it. I jokingly refer to it as a "survival guide" because (as I mention in the disclaimer paragraph) the experience is by no means a walk in the park.
Absolutely everything the GP says is spot on, the C.S. PhD is really atypical amongst all the Ph.D. disciplines. E.g. the resource constraints that exist in many disciplines don't even exist in C.S. - it is almost completely the product of thought/ideas and ever cheapening computing power. I would suggest sending out your article to Ph.D. students in different disciplines, look at all the feedback, and then incorporate them if you are so inclined. The article might look quite different.
Edit: after reading the comment from pgbovine, I felt I made an unnecessary personal comment, which I omitted. I rewrote the sentence to keep the main point.
(BTW: I guess you know http://www.pgbovine.net/PhD-memoir.htm. Also from an uber-successful PhD student, but the full story, rather than a set of advice.)
It may be something about the field (growth, competition with industry). I think it was not a coincidence that out of many friends of mine who did their PhDs, only Wojciech Zaremba (now in OpenAI) had some non-trivial impact on the world.
I don't want to imply that even if everything works (topic, advisor, funding, the sense of meaning, the sense of progress, ...) it is any easy path. And I am really sure that even with your skills, work ethics (and luck) it was a challenge. Still, even if one field is rosy (DL or maybe CS in general), a typical PhD experience is hardly sth I would recommend blindly (vide links there: https://pinboard.in/search/u:pmigdal?query=academia+depressi...).
(On an unrelated note: thank you for "The Unreasonable Effectiveness of Recurrent Neural Networks", ConvNetJS and CS231N - they brought me into the deep learning world. :))
> I do data science freelancing. That is, I take contracts related to machine learning (predicting things, e.g. user growth of a company), data visualization (custom charts in D3.js), preparing and conducting trainings in data analysis [...]
Would you mind sharing a bit on your approach to contracting in this space?
Here are a few questions: Do you do blind calls? Do you use a freelancing site? Do you work remotely? What is the typical contract, how much do you bill? Does one have to do public talks to get recognised? How much do clients value your having a PhD? How do you animate the networking? Do clients find you, or do you find them? Why are they buying, FOMO on a marketing dataset, or just plain curiosity on the subject? If you had to specialise in one niche market, what would be, what would be your approach? Basically: what would be the steps you would take should you start only with the data science technical knowledge?
If you mail me (my website's footer), I will send you a quick&dirty summary of my path & projects. In any case, some answers:
> Do you do blind calls?
Never! But if there is an opening for a full-time position sometimes I mailed them anyway, if they are interested in some specialised contracts; sometimes they were.
> Do you use a freelancing site?
No. I followed a mailing list with freelance projects in data viz. (By far the easiest place to start, as they can SEE my previous projects and current progress.)
> Do you work remotely?
Almost entirely. But for the last few weeks I am a bit more related to a particular company, and then I prefer to be on site (easier to talk etc).
> What is the typical contract, how much do you bill?
Since it varies a lot (and I want to increase it a bit), I am not comfortable to put in publicly. Expect for things I do it is also dependent on place in which I live, particular projects, negotiation skills, project uncertainty (data science is not webdev - each project has a research part).
> Does one have to do public talks to get recognised?
Yes. I mean, maybe it is not strictly necessary, but public talks (meetups, conferences, etc) and other public activity (blog posts, running communities) helped me a lot! (But I love it anyway.)
> How much do clients value your having a PhD?
A wonderful discussion starter, never a deal-maker.
> How do you animate the networking?
(Answered above.)
> Do clients find you, or do you find them?
In the last year (or more) it's only clients who contact me, and I accept (/follow up) or decline projects.
> Why are they buying, FOMO on a marketing dataset, or just plain curiosity on the subject?
?
> If you had to specialise in one niche market, what would be, what would be your approach?
If you were an animal, which... ;)
> Basically: what would be the steps you would take should you start only with the data science technical knowledge?
http://p.migdal.pl/2016/03/15/data-science-intro-for-math-ph...
A good PhD leads to many of the nice things he describes: freedom and ownership and personal growth and all that stuff. An average PhD (or worse) leads to pretty much the opposite. Most of the superstars I know went on to do pretty much whatever they are interested in at top schools. The non-superstars (and the real lumps, like me) can easily wind up in a death spiral - where your mediocre publishing record and mediocre PhD afford entry only into 3rd-tier institutions, where you will work with worse and worse people, more or less guaranteeing steadily declining quality of work. A mediocre result more or less guarantees that you will be a low-status drone in academia, trying to wedge world-class work in with a bunch of other activities (teaching, being a glorified research assistant, and other 'service').
No-one sets out to do a bad PhD, but people need to understand that the average outcome isn't nearly as glowing as Karpathy outlines. Similarly, the outcome of going to industry also has a huge range. I found myself immediately - I mean on Day 1 - doing more pure Computer Science going to work for a startup than I had any reasonable hope of doing as a semi-failed academic, and have had a steadily improving experience subsequently (some of this stems from a rather delayed growing-up on my part, so it's not entirely a judgement on industry vs academia).
However, continuum aside, there are a LOT of PhDs produced worldwide, and a LOT of good ones, and very few particularly good academic jobs. A lot of my observations of why things are kind of shitty lie not from my own career (which was largely my fault) but from observing smarter, harder-working, more organized people a couple of rungs up and realizing what a cruddy experience they were having in the middle tier. And also thinking: "man, if I just got my shit together and worked insanely hard for a few years, I could be as unhappy as Professor XYZ".
I wouldn't say "don't do a PhD". Just "know the odds"; i.e. know how many of the positions you could see yourself doing are available vs. PhD students emerging at a similar tier, and ask yourself whether you're going to be in that percentage, and what your plans are if you aren't. I learned a lot from my PhD, even if I sucked, and it turned out my backup plan was really much better than I realized it would be.
In Europe (Germany in particular), you have to do a 2-year Masters degree which tends to be ~1.3 years of coursework and ~0.7 years of research culminating in a thesis.
Judged by the quality of your thesis, you then try to get accepted into a group for a PhD which takes 3 years on average.
In the US you go straight to a 5-year PhD program and within that, the coursework-research duration split is similar to what you find in Europe.
Why is every conversation about PHDs always cast in the light of as-opposed-to-working-for-the-man? I don't see discussions ever bring up the plethora of other life courses one can take. It is though the author sees a very clear binary: PHD or go work on fixing bugs in Gmail (or some other such cog-in-a-machine project).
Where is the discussion of starting a business? Of making your own company? Breaking free of the political shackles of academia and blazing your own path to glory?
I am all for PHDs, and all for people pursuing, and pushing, the boundaries of human knowledge. I would just like to see that discussion live on its own, without comparison, if that is possible.
* Freedom
* Ownership
* Personal freedom
...
* Getting into a PhD program: references, references, references
* Student adviser relationship
* Pre-vs-post tenure
* Impressing an adviser
* (Topic) Plays to your adviser’s interests and strengths
Edit: A few people insist on casting a float into a bool. Every job has politics, ranging from 0.01% (anonymous author of 1-man SaaS) to 99% (politician). It's not very informative to note that both are non-zero.
That's crazy. The way to get a great reference is to do outstanding work. If by 'playing politics' you mean not pissing people off by being an asshole or being unreliable, then, yeah, take care to be a good colleague. But almost entirely: do really good work.
Yeah, internal corporate politics never gets people's entire project canceled with no notice in the corporate world. That would just be mean, so it doesn't happen /s.
(Guess what I found out last Friday?)
Haha, a geeky yet very apt expression.
Good point. Several, nice examples from my field of study are AbsInt, Kesterel, Galois, and AdaCore.
https://www.absint.com/products.htm
http://www.kestrel.edu/home/about.html
http://www.adacore.com/academia/projects
AbsInt turns elite work of Ph.D.'s into commercial products and enhances them. Doubt it's boring. Kesterel has people doing academic-style research on hard problems and applying it to real-world. Galois does that too. AdaCore does a combo of tooling for grunt work, advanced tooling (esp compiler), and cutting-edge work (eg SPARK provers) involving academics remote or in-sourced. A good PhD in relevant subjects might get a job at any of these companies with similarly interesting work. Or, if tool is good enough, make a company out of it by partnering with people good at the business/sales side.
So, definitely opportunities here beyond black and white of academic-only or grunt work.
The discussion of what now? Not everyone wants to start a business. Not everyone has the domain and market knowledge to start a successful business. "Blazing your own path to glory" fails 90% of the time, usually repeatedly.
If you're gonna be that damned persistent, you might as well have the freedom of academia.
I dont think man has ever been as truly free as we are today, you can really do anything you set your mind to, not confined to the short, somewhat stuffy list you and I have walked over together.
Eventually I decided that I wanted to do research and I couldn't be more happy about it. My experience as a PhD student was awesome, I did a postdoc on a topic that interested me very much but quite different than what I did during my PhD, and I've just landed a permanent position in academia as an associate professor!
Karpathy is pretty famous for his blogging and open software. His code and blog post about recurrent neural networks practically got him a language model named after him.
He also recommend you to release code in this article, so I'm not really sure what you are getting at.
Yes, he's famous for that, but that didn't earn him his PhD nor would it have gotten him tenure. As he remarks about his teaching of his course, him doing a good job actively worked against him because... it's not writing a sexy new paper or networking.
> He also recommend you to release code in this article, so I'm not really sure what you are getting at.
Releasing code isn't the same thing as creating polished end-user applicable stuff on the level of char-rnn. In ML, you're increasingly expected to at least chuck over the wall a barebones implementation to demonstrate it works at all, but there is no expectation that it will be generalized, well-written, or polished, or maintained, and typically they are not. (Most ML releases I've looked at are kind of horrifying from a software engineering perspective. Just thinking about improved-gan makes me shudder.)
I haven't read that. The author says that poorly incremental work is discouraged.
> Get into only the elite colleges and betwork at conferences as much as possible
It's about being part of the community, having brilliant people around you to push you and exchange ideas with, meeting people to collaborate with...
> Personal freedom. As a PhD student you’re your own boss. Want to sleep in today? Sure.
This is largely true, but only if you're on good terms with your advisor and they're happy with your progress. God, I miss being able to sleep in until 2pm.
> Personal growth. ... you’ll become a master of managing your own psychology
Yes, it's definitely a roller coaster. I know what happens to your body after a month when your only calorie source is peanut butter and white bread. Depression, random trips to Canada, and more.
> Picking the school. ... your dream school should 1) be a top school
No, at least, in mine and other's experience, you should go to the best school where you're still capable of being in the top ~1% of your graduating class. You'll feel like you're the best and that's almost all that matters (Malcolm Gladwell's talk [1]).
> So you’ve entered a PhD program and found an adviser. Now what do you work on?
You will not be interested in the same exact topic for ~5 years straight, so keep that mind. Try to keep it broad.
> Giving talks
Do this / practice this as often as possible. It's how you'll get hired (or not). I've had to sit through some embarrassingly bad ones, where the candidate then has to survive the next 7-hours of interviewing where everyone knows they're not getting hired. (In my experience the talk is first thing in the morning.)
[0] He gave the fields of "Computer Science / Machine Learning / Computer Vision research" as a disclaimer, my disclaimer is that I only know about experiences in molecular biology / chemistry / materials science / synthetic biology / microbiology. [1] https://www.youtube.com/watch?v=3UEwbRWFZVc
> No, at least, in mine and other's experience, you should go to the best school where you're still capable of being in the top ~1% of your graduating class. You'll feel like you're the best and that's almost all that matters (Malcolm Gladwell's talk [1]).
Isn't that a bit short-sighted? What happens after you want to progress to the next environment?
Is it realistic to be in the top 1% anywhere? Isn't that like being the best graduate ever from a small program, or the best in a decade at a larger one? You're not competing with a large population anymore, you're competing with someone who has passed dozens of filter steps in their lifetime and have gotten just as far as you have.
It does sound odd, or just wrong. But from Gladwell's example [0], and others, top students in their class, regardless of the school, perform better than expected following graduation. From my experience, that's at least partially due to the increased attention (and better training) that top students receive. The lower ranked students are more-or-less ignored and pushed out.
> Is it realistic to be in the top 1% anywhere?
No. But you may have a higher chance of being a top student at a cheap in-school school than at Harvard.
[0] https://ideas.repec.org/p/van/wpaper/vuecon-sub-13-00009.htm...
If anything, being in one of the few labs that work on "top" things in a specific domain pretty much guarantees that you'll be in that 1% of your subfield; and if your lab is weak in that (though possibly world class in other fields), then you won't.
Many universities have a "College of Science." Many of those universities give out some sort of "Top Doctoral Student" award, with a different name.
And I largely agree with what you say. That would be how any rational person would measure themselves, you would think. But in reality, you're comparing yourself to the people you're surrounded by, not with the smart guy in Iceland.
I can potentially see how your advice could work for undergrad, but doing good research is difficult if your peers aren't as good. Your discussions are less interesting, your ambitions are scaled down, and so on.
> Exclusivity. There are very few people who make it to the top PhD programs.
Top companies are even more exclusive than PhD programs in terms of acceptance rate.
> you’re strictly more hirable as a PhD graduate or even as a PhD dropout and many companies might be willing to put you in a more interesting position or with a higher starting salary
This is 100% false. Many many people have found a PhD to be a handicap when it comes to getting a job, particularly in software engineering. A large number of employers have anti-PhD biases which will work against you.
> Ownership. The research you produce will be yours as an individual. Your accomplishments will have your name attached to them.
My experience with academic research is just the opposite. I think it's patently ridiculous that professors get "authorship" credit on papers even when they had a minimal, at best, role in it. Meanwhile in companies you can have a tangible impact and see real results/credit from it (bonuses, promotions). Not to mention that many universities have draconian IP policies.
Acceptance rate is meaningless unless the base populations are the same. Considering only the (hopefully) top graduates are interested in applying for PhD, whereas most graduates have to look for jobs, acceptance rates between grad school and companies are really comparable.
> I think it's patently ridiculous that professors get "authorship" credit on papers even when they had a minimal, at best, role in it.
Think of professor as a spiritual mentor. I don't think Jony Ive put his hands on the drawing of latest apple watches anymore, and he gets to sit in the white room every year. It's the philosophical guides he puts into his team that earn him the privilege.
In terms of material compensation, I totally agree with you that companies generally pay much better. The credit part is really nominal. I've know some people working in google, facebook. Their projects range from google cardboard, google daydream, fb live video, to fb 360 video. Honestly, I have never seen their names/credits anywhere else but our private conversations. It's nowhere to be found.
> This is 100% false. Many many people have found a PhD to be a handicap when it comes to getting a job, particularly in software engineering. A large number of employers have anti-PhD biases which will work against you.
I was wondering if you could comment on this? Personal experience? I may be naive for not believing this in the first place; why would there be disdain expressed towards PhD applicants at a company? Is it a business perspective ("PhDs are too expensive"), technical ("they are too specialized, cannot practically implement solutions we expect of a new hire"), social ("I don't understand academia and couldn't achieve that high"), or something else?
I do not have a PhD. My experience is based on personal experience with hiring people, speaking to friends and other hiring managers, and anecdotes from HN.
In general, the reasons are (for better or worse):
1. PhDs aren't very good programmers or don't follow software engineering best practices.
2. PhDs want to do "research" and will get bored with the basic software production required for 90% of industry jobs.
3. PhDs expect to be paid/respected at a higher level of seniority, even though skill-wise they'e often barely above a recent BA graduate.
I don't know how valid all of these are, but in general I would personally always choose someone with 5 years of industry work experience over someone with a PhD for a software job.
False. Do you even know how many people apply online to Uber, Slack, Facebook, and Google? More than the entire population of flagship state universities.
> > you’re strictly more hirable as a PhD graduate or even as a PhD dropout and many companies might be willing to put you in a more interesting position or with a higher starting salary
> This is 100% false. Many many people have found a PhD to be a handicap when it comes to getting a job, particularly in software engineering. A large number of employers have anti-PhD biases which will work against you.
I agree.
> Meanwhile in companies you can have a tangible impact and see real results/credit from it (bonuses, promotions). Not to mention that many universities have draconian IP policies.
The results come from products you did not solely produce. There are plenty of other co-workers or teams which helped contribute large components to your product.
I'm not sure if you're agreeing or disagreeing. That's precisely my point: Google, for example, gets tens of thousands of applications. Their acceptance rate is incredibly low.
> The results come from products you did not solely produce.
Sure, but I don't see how that detracts from the fact that you can see meaningful rewards for your work in corporate life.
Experiences differ. A lot. Personally, I had an awful time during my PhD, and between the penury and the toil and the bleak prospects afterward, I'd say you're definitely in the minority on this one.
I wouldn't recommend a PhD to anyone who isn't dead set on a job that requires one.
Do you have any evidence for the minority claim beyond the personal anecdote? Also, this is field dependent, so a mention of the field could be helpful.
Mine experience was completely driven by my choice of advisor. He did not push me and therefore I had complete freedom. I was glad to have the funding through him, but did not get much direction. I did almost everything.
And while it took me 7 years to complete, I don't look back on the experience as only "getting my Phd". I learned so much about myself; I traveled around the world; I received my private pilots license; I learned about cars; I played a lot of golf; I had a great time. There were some shitty times where I had to push through, but I have no regrets. Most of my friends are buying houses and having kids, so I'm a little behind on my career/savings. But hey, I have three degrees in a pretty good field and my career is full steam ahead.
http://blogs.lse.ac.uk/impactofsocialsciences/2013/12/11/how...
This seems unfair. Many otherwise good students don't get taught this coded language. I understand that heuristic or incremental developments might not be accepted at top conferences, but the work should be judged on what it does rather than the inexact word choice of a student. It feels a bit cliquey.
Almost no one is taught this coded language. You absorb it by reading tons and tons of papers, then more papers. Karpathy is unusual and fun to make it explicit like this. He's a canny person.
So much of the crap that I see people complaining about in the sciences simply doesn't occur in the humanities. Some of it I wish would (co-authorship would be a good way for those of us in the humanities to learn how to write an article for publication) but I am glad I didn't have to deal with a lot of the lab and advisor drama that I have seen (or arguing if someone should be fifth author or sixth...).
Of course I was making slightly more than half my peers in the sciences at the same university were making and they have a lot more career prospects than I do so maybe it is worth it....
Coauthorship IME has always been extremely amiable. Often, when I am a middle author, I almost feel that I'm being done a favor by being put as a coauthor when I didn't do that much work (but I did make a material contribution). Conversely, as a first or senior author, it costs you nothing to add coauthors and gains you goodwill. It's win-win. It's very easy to get coauthorships if you keep your eyes open.
In my experience, if you want to succeed as a Ph.D. in the sciences you have to find a niche that puts you in the position of being useful to other people rather than in competition with them. If you don't, you will fail. If you do, it is almost comically easy to be at least moderately successful.
My experience of getting the PhD was pretty positive (finished after three and half years, good university, good subject, great supervisor) but I still see so much truth in these essays: http://100rsns.blogspot.co.uk/
Or to be more succinct, https://en.wikipedia.org/wiki/Sayre%27s_law
Personally, I finished the PhD in May and moved on to a really fantastic job in my field. I have time and money that I wouldn't have as an academic.
I'd like to clear up a few misconceptions I had about the academic lifestyle going in. This is about the professor job, not grad school.
1. Being a professor is not about teaching. I went into grad school thinking I could make a difference. Address the rampant gaps in my colleagues' education. It turns out that professors generally abhor teaching, and that departments actually use teaching assignments to punish underperforming faculty. Performance means papers and even more importantly grant money. Teaching doesn't enter into it.
2. Professors do not have any free time. Even tenured ones. That appealing academic calendar is a mirage. If you are a professor, you're a professor every waking minute. You spend your vacations at conferences, your summers trying to get ahead on research. You read papers before breakfast and after dinner. Or you burn out after getting tenure, get a heavy teaching load as punishment for your lack of productivity, and turn into the kind of professor everybody has an anecdote about. The one with nutjob politics who never turns up for office hours, or the one who lectures while hung over or still drunk.
Sorry to rant, grad school is no picnic and I wish I'd gone into it with my eyes wider open. I'm happy with the end result, personal growth, seeing things on a higher level, being "doctor" so-and-so, having an awesome job, but it's not at all an easy way out if you're tired of what you're doing now.
I guess what you should learn depends on the field. But they'll be looking for demonstrable interest and possibly outside learning.
I literally cold-emailed some potential advisors that I found from looking at university websites and told them what I was passionate about. I got asked in for an interview and pitched what I wanted to work on.
I think if you are grown up and passionate about doing a PhD you are already going to look pretty impressive, as most advisors will usually be dealing with young undergraduates who can often be drifting into a PhD without really knowing what they want.
Someone dumping a paying career they have have already invested most of their 20s into has obviously thought about it very carefully and knows what they want.
It practically is a litmus test on answering what you should do, not necessarily in research, but in life.
For those interested, Hamming course on "You and your research" is on Youtube and has a _ton_ of _practical_ advice to future engineers.
perhaps instead of discounting his experience... it would be better to take his advice
Ha, sarcasm, very funny. Seriously though, outside of academia, if you're at an actual cocktail party or something with normal people, don't ever say that you have a PhD. It's like admitting that you're into some really weird shit. If anyone else mentions it about you, quickly change the subject to something less cringeworthy, like herpes.
And for the love of God, never ever call yourself a "doctor." WTF is wrong with you? No.
If you're still a graduate student ("working towards"), this is all moot, because they shouldn't have let you into the party.
You get around USD 70k per year in Switzerland for a Computer Science PhD in Switzerland (at ETHZ or EPFL). You could get up to five times this amount working in the industry, but usually not much more than that, especially when you have just graduated. It is definitely worth it to take the salary into account when you choose which universities to apply to for a PhD.
Yeah, I've heard someone - Euclid may be? - said something similar about the road to science... :)
Love Andrej but really, this isn't a common experience.