1. minGPT, nanoGPT (transformers)
2. NLP (make more series)
3. tokenizers (his youtube)
4. RNN (from his blog)
There are many domains which don't have a karpathy and we don't hear about them. So glad we have this guy to spread his intuitions on ML.
Whereas for example Jeremy Howard's style resonates a lot more with how I enjoy learning, very much a "let's build it" and then tinker around to gain intuition on how things inside the box are working.
I see the benefit in both approaches and perhaps Karpathy is more methodical and robust. But I just find Howard's top-down style a lot easier to stay motivated with when I am learning on my own time.
So I say, build the thing, figure out where the shortcomings in your knowledge are, and continue refining. One of those things will inevitably be math. Maybe it will be signals processing the next week or fundamentals of Spark the next. And there are always interesting papers coming out.
Karpathy's style, for me is more like at the right abstraction to bring out curiosity in me towards the subject. After watching his lectures, i go on to more materials generally, and never really stop there.
He is for sure a cool guy.
1. technical track (all the GPT repro series)
2. general audience track
For (2), I had a 1hr video from 1 year ago, but I didn't actually expect that video to be some kind of authoritative introduction to LLMs. The history is that I was invited to give an LLM talk (to general audience), prepared some random slides for a day, gave the talk, and then re-recorded the talk in my hotel room later in a single take, and that become the video. It was quite random and haphazard. So I wanted to loop back around more formally and do a more comprehensive intro to LLMs for general audience; Something I could for example give to my parents, or a friend who uses ChatGPT all the time and is interested in it, but doesn't have the technical background to go through my videos in (1). That's this video.
1. https://github.com/mistralai/mistral-inference/blob/main/src...
Not for nothing, Andrej's parents maybe different from mine, but I definitely can't send this to my parents.
I have one "Intro to LLMs" video already from ~year ago, but that is just a re-recording of a random talk, so I wanted to loop around and do a lot more comprehensive version.
If you want to start getting your focus back under control, give meditation a try. It's a gentle tool to will help you get you understand how your attention works, and, with training, will give you back the control you need.
I also find I can watch a good bit of a complicated/technical video while walking on a treadmill/stairmaster at the gym. Just enough noise of other people, and the determination to do 45-60 minutes on a fitness machine replaces the determination required to get through a video. Doing it 3 days in a row feeds the motivation cycle in me. After day 1 I'm itching to get back to the gym and get back to the video to do another "episode".
I think this is important content, the more people know how ai works under the hood the more empowered society will be.
We know from some of the leaked Musk/OpenAI emails they didn't really believe it internally and thought they would need to merge with OpenAI to do it.