THE AI READING LIST
Ilya’s 30 papers as audio
If you really learn all of these, you’ll know 90% of what matters today.
- 01PaperAudio ready
- 02PaperAudio ready
- 03PaperAudio readyGPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism
Yanping Huang et al. (Google) · 2018
Paper - 04PaperAudio ready
- 05PaperAudio ready
- 06PaperAudio ready
- 07PaperAudio readyIdentity Mappings in Deep Residual Networks
Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun · 2016
Paper - 08PaperAudio ready
- 09ArticleAudio ready
- 10ArticleAudio ready
- 11PaperAudio readyDeep Residual Learning for Image Recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun · 2015
Paper - 12PaperAudio ready
- 13PaperAudio readyDeep Speech 2: End-to-End Speech Recognition in English and Mandarin
Dario Amodei et al. (Baidu Research) · 2015
Paper - 14PaperAudio readyOrder Matters: Sequence to Sequence for Sets
Oriol Vinyals, Samy Bengio, Manjunath Kudlur · 2015
Paper - 15PaperAudio ready
- 16PaperAudio ready
- 17PaperAudio readyNeural Machine Translation by Jointly Learning to Align and Translate
Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio · 2014
Paper - 18PaperAudio ready
- 19PaperAudio readyQuantifying the Rise and Fall of Complexity in Closed Systems: The Coffee Automaton
Scott Aaronson, Sean Carroll, Lauren Ouellette · 2014
Paper - 20PaperAudio readyImageNet Classification with Deep Convolutional Neural Networks
Alex Krizhevsky, Ilya Sutskever, Geoffrey Hinton · 2012
Paper - 21ArticleAudio ready
- 22PaperAudio readyKeeping Neural Networks Simple by Minimizing the Description Length of the Weights
Geoffrey Hinton, Drew van Camp · 1993
Paper
ALSO ON THE LIST
Read the remaining works at the source
These courses, books, and code-heavy pieces do not currently have a ListenDock audio episode.
The Annotated Transformer
Sasha Rush et al. (Harvard NLP) · 2018
Kolmogorov Complexity and Algorithmic Randomness
A. Shen, V. Uspensky, N. Vereshchagin · 2017
CS231n: Convolutional Neural Networks for Visual Recognition
Andrej Karpathy, Fei-Fei Li et al. (Stanford) · 2016
Machine Super Intelligence
Shane Legg · 2008
A Tutorial Introduction to the Minimum Description Length Principle
Peter Grünwald · 2004
THE STORY
When legendary game programmer John Carmack decided to move into AI, he asked OpenAI co-founder Ilya Sutskever what he should read. Ilya handed him a list of around thirty works and said that learning them would cover 90% of what matters in modern deep learning.
The version circulated today was reconstructed by the community. It is a coherent tour from convolutional and recurrent nets, through attention and Transformers, to scaling laws and the information-theoretic roots of learning.
Sources: community reading list, Aman’s AI Journal, and Ilya’s List.
Have a paper of your own?
Upload any PDF or paste a link and ListenDock turns it into a clear audio episode.
Turn a paper into audio