The only thing I'd probably add is that there's a pretty significant gap going from learning linear algebra to more advanced topics such as LDA.
For people who are just getting started with machine learning, it's probably best to get started with implementing some of the more "intuitive" algorithms such as decision trees, k-means, and naive Bayes before moving over to some of the more recent academic work.
Other things that are pretty useful, but often forgotten, such as feature selection, data normalization, and even data visualization. Algorithms are usually just one part of machine learning, but even the best algorithm wouldn't be able to do anything without identifying what the best features of your data are.
Still, it's a great list of more advanced topics, and definitely something I'll keep bookmarked for future reference.
For LDA you'll need to understand Dirichlet processes, I find the introduction by Frigyik et al. [2] to be excellent for that. You may need to read A Measure Theory Tutorial (Measure Theory for Dummies) by Gupta [3] before. Finally, I put there the two most influential LDA papers to me: [4] and then [5].
[1] http://www.inference.phy.cam.ac.uk/mackay/itila/book.html
[2] http://www.ee.washington.edu/research/guptalab/publications/...
[3] https://www.ee.washington.edu/techsite/papers/documents/UWEE...
[4] http://www.psychology.adelaide.edu.au/personalpages/staff/si...
[5] http://videolectures.net/site/normal_dl/tag=83534/nips2010_1...
For some of the non-parametric variants like hierarchical dirichlet process LDA, you need DPs, but that stuff is pretty hardcore -- don't walk before you can run.
Another route to LDA (assumes some Bayesian stats basics):
* Learn a bit about Markov chains if you don't know them already * Read up on sampling-based approximate inference methods and find a proof that a Gibbs sampler converges (or just take it on trust...) * Read the classic Griffiths and Steyvers paper deriving a collapsed Gibbs sampler for LDA [1]
I was wondering, could you list some of the more recent academic works? I've touched up on the basics and feel pretty comfortable with them so I want to try something a bit more advanced.
best machine learning site:stackoverflow.com "closed as not"
It's important to understand individual algorithms, but in many ways it's more important to have a broad overview of the field and its more modern methods, so that given a problem it's possible to think about the best way to solve it, and to share a common language with others who may have ideas. Beyond this list and various online courses, I've found that talking to people about their work and explain the high-level concepts of every black-box classifier or similarity metric or whatever it is they use has been quite educational
I did note the absence of the oft quoted Andrew Ng's Coursera course on ML. I assume the author has put it under : "disruptive educational sites".
But genuinely want to know how Ng's course measures up to the other resources mentioned in this post??
