Some resources to start out with (definitely worth going down other interesting rabbit holes!).
- Andrej Karpathy — watching many of his videos (includes "GPT from scratch")
- Pieter Abbeel lectures
- MIT AI lectures
- Scaling talk (Stanford Online)
- Watching some of Yannic Kilcher's videos is also useful
- How to read papers (fast.ai forum)
- Ilya Sutskever list of papers/topics
- Reddit thread: "The ML papers that rocked our world 2020–2023"
- ResNet
- VAEs
- GANs
- Transformers
- VQ-VAE
- GPT1
- GPT2
- Vision Transformers
- Scaling Laws
- GPT3
- Chinchilla (compute-optimal scaling)
- Diffusion explainer — Umar Jamil
- Jeremy Howard talk (part 1)
- Jeremy Howard talk (part 2)
- Denoising Diffusion Probabilistic Models
- Latent Diffusion (LDMs)
- Diffusion Transformers (DiTs)
- MIT Diffusion models intro
- Flow = Diffusion
- Flow, OT, and rectified flow explained simply
- Energy-Based Transformers covered by yannic kilcher
- Energy-Based Transformers covered by bycloud
- Energy-Based Transformers covered by AI Coffee Break
- Energy-Based Transformers, (see sections H and I for an easy to understand intro)
- EBT Blog
- Implicit Generation and Generalization
- Iterative reasoning as energy minimization
- Iterative reasoning as energy diffusion
- EBMs for compositional generalization
- Karpathy — debugging/training ("Recipe")
- Google Research — hparam tuning playbook
- Lilian Weng
- Tim Dettmers
- Karpathy (home)
- The Bitter Lesson
- Coding models in PyTorch (stable diffusion, LLM transformer)
- Code DiT architecture from scratch based on the paper using only the trainer and diffusion module (no using LLMs for help)
Twitter / X — people to follow (unfortunately, most of AI informal discourse happens on social media)
- Yannic Kilcher
- Yann LeCun
- AK (@_akhaliq) — daily paper drops
- Allen AI
- Chris Paxton
- Jim Fan
- Zeyuan Allen-Zhu
- Lucas Beyer (@giffmana)
- Clément Delangue (HuggingFace)
- Andrej Karpathy
- Jeremy Howard
- Lilian Weng
- Soumith Chintala (PyTorch)
- Aran Komatsuzaki — paper threads
- Sasha Rush
- Hardmaru (David Ha)
- Sander Dieleman — great on diffusion
- Jürgen Schmidhuber
- Tim Dettmers
- Yannic Kilcher — paper walkthroughs
- bycloud — research news roundups
- 3Blue1Brown — math intuition
- AI Explained
- Fireship
- Jia-Bin Huang
- Steve Brunton (Eigensteve)
- StatQuest
- Andrej Karpathy
- Deepia
- Two Minute Papers
- Machine Learning Street Talk — long-form interviews
- AI Coffee Break with Letitia
- Computerphile
- Sentdex
- Sign up for HuggingFace Papers daily emails — curated top arxiv papers
- bycloud newsletter — weekly AI research roundup
- Rewatch/re-read things you didn't get initially after some time — progress is real but gradual.
- Use LLMs to help explain papers and math (just be careful doing so)
- My blog on how to do AI research (coming soon — feel free to ask me for the draft)