I expected that around 10,000 people would have a perfect score in the class given my experience with Asian participation in online programming contests. There are a few countries in that part of the world where organizing a group to check each others' work and ensuring perfect scores is common and encouraged even when the official requirement is to do your own work. And those happen to be countries filled with millions of smart people. India, China, and Russia seem to be the heart of the phenomenon. Google, Facebook, and Topcoder have systems to deal with it but ai-class.org does not.
Anyway I'm neither surprised nor disappointed not to be in the top sliver. I'm not hireable anyway. It's funny that there is an identifiable top 1,000 at all instead of 10k+ perfect scores.
How do you know this?
Why? Because it is interesting. Find something to apply to that might add value to the business. If a enough people do that, AI techniques might become common place. Get on Kaggle, GitHub your next project etc. Putting what we've learned to practical use is how we can proliferate and disseminate this knowledge. This was the intent of the class in the first place anyway.
tl;dr Using this course as a gateway to your next job is shortsighted IMO.
You can't sit back and wait for AI to "happen". What would that even look like? The benefit of these techniques is in application to existing data; you don't have to quit and found some kind of computer vision/kinect/big data startup to take advantage of the knowledge.
You should be aiming to put yourself out of a job by using AI.
edit: why is this getting voted down? I would expect there are a fair number of people here who took the course with a casual approach, like I did. E.g., I usually watched the lectures early Sunday, then did the homework that night. Then, for those topics that I enjoyed and wanted in more depth, I read the sections in the book on them. This approach is relaxing and fun, but does increase the tendency to make silly mistakes and not catch them (which I in fact have done a few times now).
I am, it turns out, enjoying the material enough that I think I would like, if I ever find myself looking for a job, to find one where I can use this stuff.
(Lest I seem overly harsh, I'm in the same boat.)
Basically your answer is like the classic "I could have got a 5.0, but I partied", which is a statement that says a number of things. 1. "I think all the kids who got 5.0 are boring and don't party" (untrue). 2. "I am not capable of getting a 5.0 and partying" (probably true).
That the person would even make the statement says something about their general level of intelligence. Hence when you say "If I had but known, I would have been in the top 100", you are saying "I had no idea that a hugely publicized course at a renowned university might lead to job opportunities". This lack of intelligence seems to contradict your first statement. Or it could be that you are not in the habit of thinking about such things. I hope you are either a kid, or an incredibly wealthy trust-fund baby living in isolation. Usually, however, I hear such remarks from people who are not smart enough to realize they are not smart enough, or they are quitters, or they are victims.
The "partying" I was doing instead of putting a non-casual effort into the class was dealing with some projects at work, and starting to read a book on analytic number theory because I'm tired of being someone who doesn't understand the proof of the prime number theorem.
> you are saying "I had no idea that a hugely publicized course at a renowned university might lead to job opportunities"
Yeah, I did not think it would lead to much in the way of job opportunities. That's because the class is just an introduction. If you compare almost any section of the class to the corresponding section in the Russell and Norvig book, you'll see that they are leaving out a lot of detail in the class. The homework in the class almost never pushes any limits of what was done in the lectures. Many of the homework problems are just things that were done in lecture with the numbers changed.
I would have expected the job opportunities to start at a deeper level than the class reaches.
I presume people are guessing how they are doing by just looking at their homework and midterm scores so far, weighted according to the appropriate weights.
That's at least partly due to an extremely effective teaching system and style, so I'd say it's a worthwhile tradeoff.
Of course, I know it's their first class and you can't get everything perfect right off the bat. But I'd love to see the lab assignments being one larger project, some kind of task where you need to apply the principles learned to solve different problems. It would be also more effective in measuring/grading students.
On an unrelated note: I wonder what is the license for the classes/videos. I am working now in an adaptive e-learning platform. It would be cool to put their videos and questions in our system and see how effective the system is.
The course was awesome for what it was: a chance for the average programmer to get their feet wet in ML, but the fact that it was hosted by Stanford seems to lead to a somewhat humorous irony that it was pretty accessible whereas Stanford's real-life rigor is supposed to be nothing of the sort.
However, I doubt they have the bandwidth to manually grade thousands of assignments. Given the scale of the class, the grading is forces to be automated. It's hard to come up with such assignments and even harder to make them challenging, given an automated grading script.
So if ML numbers are in any way similar to this maybe it wouldn't be so bad. That is if there was such a letter.
I'll add that Professor Widom's style is quite engaging as well.
Personally I've always got a bit wrong in most of he homeworks but usually felt that while I understood the material I didn't recite enough time to rigorously check my answers and rewatch the materials to pick up on some of the subtle things you need to remember or some questions.
The correlation will be interesting. Undoubtedly the top 1000 is made of many of the type of people that must be the best at whatever they try (not exclusively of course) and with a decent amount of time on their hands, after all there is no disincentive whatsoever for not completing something and no previously known reward for perfection (I guess a piece of paper can count). Given some wording in some of the Qs (actually my only complaint about this course) a lot of care must be taken to ensure that every question is right, so being in the top 1000 is no small effort.
That said I have a job I like already, and plan on using what I learned here.
Those 1000 with more time and drive to be the best probably are some of the best job candidates though. Plus chances are they do need a job after all.
So their metric isn't ideal for identifying passionate poeple ;-)
What struck me was how much importance he gave to this metric, which isn't that hard to game on an online offering.
I know a few people (who shall remain nameless) who collaborate and check each others answers and so on before submission, in direct violation of the Stanford policy (and have 100s or close to it), and so probably have received this mail, whereas more "deserving" (note quotes) people who honestly work through the course material may not because they have, say, a 85% or 90% score.
That said, my key takeaway from this is that professors are very impressed by perfect scores irrespective of how you got them. There must be something magical about that row of 100s. Once you set up a grading/ranking system, it is psychologically very hard not to admire people who end up at the top.
I am personally a little dubious that the people with the highest scores would make the best pool of employees, especially given that this is an online course without the programming component, but what do I know?
I wrote Java code for most of the algorithms in AIMA as a side project a few years ago [1], and after I read an online post by Peter Norvig saying a few of his students had tried and failed a few times (to implement the code in Java- Common Lisp code existed and the Python version was in its infancy), I sent him the code and this became the "official" java distribution for AIMA ( though I don't maintain it anymore- the immensely talented Ciaran O'Reilly of the Stanford Research Institute does) and no one ever invited me to Stanford or offered me a cool AI job[2], sob! :-P.
No I am not bitter I tell you, not even the teeniest bit :-p [3]
I wonder how this signalling will play into the upcoming courses? If there are tangential real world benefits to be gained by attempting a "perfect" score, then you can expect a lot more game playing wrt scores and exams.
[1] More about how Peter Norvig shredded my initial code etc here http://news.ycombinator.com/item?id=2405277
[2] though eventually, after a lot more work, it did lead to my working on good ML/robotics etc projects from Bangalore, which is a hard thing to do in the Great Outsourcing Wasteland.
[3] I am really not bitter.
I wrote the code for the hell of it, not to get a job. AIMA was my introduction to the fascinating field of AI. It is a great, great book and it has a lot more material than is covered in the course.
I once did want to go to Stanford and learn from the great profs there, but now in a "mountain comes to Mohammed" fashion, Stanford is coming to me. I don't care about the credentialling - I just want to learn. I took the AI online course and enjoyed Peter's and Sebastian's teaching immensely. Fwiw I should have a high 90's score, (I didn't add it all up) but nowhere near a perfect score.
Second, I registered for the Machine Learning course (I am not sure if the same applies to the AI course) and I compared it with the actual ML course at Stanford (CS229) (I mainly looked at Youtube videos of Andrew[1] as well as Assignments/Midterm[2]). The latter is by far more advanced and theoretical. The assignments tend to test more than basic comprehension of the material presented in the lectures, which is exactly what the online course reviews tend to evaluate. They require strong mathematical knowledge and obviously a minimum level of creativity/intelligence.
[1] http://www.youtube.com/results?search_query=machine+learning...
I don't. That part of the post was written with tongue firmly attached to cheek. If that tone didn't come through, that means I have to improve my writing.
The online ML course is CS 229A (which is also an actual course at Stanford. The online version is close to the Stanford course).
The "tough" version is CS 229 (no 'A' at the end). I registered for the ML course thinking it was an online version of CS 229 and dropped out when it was confirmed to be 229A. In my politically incorrect opinion, 229A is close to worthless. The math is important in real world ML. This course included gems such as "if you don't know what a derivative is, that is fine".
The online AI course is almost exactly the same course as Stanford (CS 221), minus, of course, the programming assignments. It is an introductory, broad based course, and it does the job well (imo)
The online DB course is almost (if not exactly) the same as Stanford CS 145. I think this was the best course of the three.
All courses track the corresponding Stanford courses.
Another interesting aspect of this "online ethics" is that there's no technical measures preventing you from pulling up past quiz results, and even if there was, you could still keep previous sessions open in a browser and even if there were countermeasures against that (javascript erasing), JS could be disabled, etc.
The Programming Projects that ML class had were slightly better metric of performance as there's more work that would have to be plagiarized, and if you're just going to go through life outsourcing all of your work then I guess that's your prerogative. However, I think that if you wanted to be very serious about actually testing for knowledge of material then the addition of some sort of interview component (phone/skype session), while time-consuming, could help.
1) you immediately know if you got it right or wrong when you submit, so you can to a lesser extent brute force the correct answer
2) with the exception of maybe the first assignment, they are all "fill in the blank" sort of programming assignments. You basically just have to find the equations they give you in the PDF, translate them directly to Octave, and bam you're done.
I personally have only scored straight-100%s in a single course (Python programming), and that was only because I was relatively an expert in the material before the course began.
The work you did strikes me as a far better measure of character and subject understanding.
Why would someone do this for a free online course that gives no credit for a degree? I mean, the whole point is to learn, not get the highest grade. I really fail to understand people sometimes.
* rather than just giving up on a problem, you can talk it out and learn together * you get the opportunity to teach the material that you think you know that others find hard (a good heuristic for problems you may have just barely understood, but gotten correct anyway). Teaching material is a great way to learn it, and expose any gaps you might have in your knowledge. * instant feedback on problems while they are still fresh in your memory
Your final score will be calculated as 30% of the score on the top 6 of your 8 homework assignments, 30% your score on the midterm exam, and 40% your score on the final exam. For those completing the advanced track you will receive your final score as a percentage as well as your percentile ranking within all those who completed the advanced track, and this will appear on your statement of accomplishment. The statement of accomplishment will be sent via e-mail and signed by Sebastian Thrun and Peter Norvig. We hope to have them digitally signed to verify their authenticity. It will not be issued by Stanford University.
Its wonderful that Sebastian & Peter are reaching out like this after all they have done so far. Congrats to everyone that slugged it out, and good luck on the final this weekend!
They will also probably look closely at that optional programming assignment as well.
I knew there was a reason Google let Peter Norvig take the time to do this ;)
Follow up is how we feel about that...
On the other hand, it was inevitable that as long as "grades" or equivalent are officially certified (even though the course is not for degree credit), that people would collect online courses as credentials. Wherever there's signaling value, there will be cheaters faking the signal.
Disclaimer: I got the email and missed two homework questions.
I made 6 100's, a 98, and a 97, and a 100 on the midterm and I got it.
:(
I was one of three researchers in a team at Yorktown Heights that did some of the best research in AI in the world, published a string of peer-reviewed papers, was the source of two commercial products, gave a paper at the AAAI IAAI conference in Stanford with for that year the 25 best AI applications in the world, personally won an award, etc.
I've taught computer science at Georgetown University and Ohio State University. My Ph.D. is in applied math.
Once in my career, just as a 'scientific programmer', in two weeks I sent a few resume copies, went on seven interviews, and got five offers.
But after my Ph.D. and AI work, I sent over 1000 resumes to Google, Microsoft, GE, FedEx, and hundreds more, got only five interviews, and no offers. I got a nice letter back from Fisher Black (as in Black-Scholes) saying that he saw no applications of applied math or AI at GS.
I ask you: Who will hire you in AI and why?
In business, hiring is because some manager has some work to do and a budget to do it. That manager believes that they know nearly all that is needed to do the work and otherwise would not be betting his career on the work. Thus, the manager is not hiring high technical expertise he doesn't have. Instead the manager is, as on a factory floor 100 years ago, hiring labor to add 'muscle' to his work.
In particular, unless the manager knows AI, he won't be hiring for AI. And there is at most only a tiny chance that the manager knows AI and even less chance that his project will depend on AI.
Moreover, the manager does not want competition from below and does not want his project 'disrupted' from below so really doesn't want technical expertise above what is needed just to get his project done.
Net, if you know some AI and want to use it in business, then find an application and start your own business. Then, since you know AI, you won't have to hire anyone in AI either.
What I've said here for AI holds for essentially all advanced academic topics.
Sorry 'bout that.
So, since this thread was about 'job placement' for students who did well in an AI class, I posted about my experience getting hired where part of my background was some expertise in AI.
Net, I have to conclude that, in getting a "job", expertise in AI will be from very rarely helpful to often a serious disqualification as in "It appears that you are overqualified for our position and would not be happy in it".
Edit: for example some opportunities here http://www.kdnuggets.com/jobs/
The main issues were (1) I am a US citizen and, thus, not on an H1B and (2) I was over 50.
Generally a subordinate is supposed to be younger.
I'm at a 4-person startup and AI is the core of what we do. My e-mail is on my profile if you (or others) are interested.
You sound like a qualified candidate, if only you'd sent your resume our way.
Sorry 'bout that.
Finally I gave up on getting hired and decided to start a business.
I'm now deep into that effort and at least for now should continue it.
