xander.ai
Getting Up to Speed on LLMs and AGI
All of the podcast episodes and videos I went through to gain a mental map of the field.
Xander Dunn, 16 Jan 2023
When I decided to focus on language models and paths toward general intelligence I found it useful to gain a map of the people who’ve developed the state of the art by listening to them.
Conversations over a few years old are generally not very relevant given how quickly the field has changed, so I skipped most older conversations.
This is about learning the vocabulary, the state of the art, what the people are like, and where they think their work is headed. Of course technical details and actually how to do these things is going to require a lot of engineering and paper reading that is definitely not contained within these.
In no particular order.
Podcasts
Should be able to find any of these on your favorite podcast app, such as Overcast, Apple Podcasts, Spotify, Audible, etc.
- Oriol Vinyals on Lex Fridman, Research Scientist at DeepMind
- Oriol Vinyals on Eye On A.I.
- John Carmack on Lex Fridman, Founder of Keen Technologies to make AGI
- Yann LeCun on Lex Fridman
- Yann LeCun and Daniel Khaneman on Big Technology Podcast
- Yann LeCun on Big Technology Podcast, 2023
- Reid Hoffman + Sam Altman
- Andrej Karpathy on Lex Fridman
- Mira Murati on Greymatter, CTO of OpenAI
- John Schulman on TalkRL, Cofounder of OpenAI
- John Schulman on The Thesis Review
- Daniela and Dario Amodei on Future Life, Cofounders of Anthropic
- Wojciech Zaremba on Lex Fridman, Cofounder of OpenAI
- Wojciech Zaremba on W&B
- Scott Aaronson on Lex Fridman, Quantum Computing Professor interested in AI safety
- Holden Karnofsky on Lunar Society, $30M check in OpenAI in 2016
- Demis Hassabis on Lex Fridman
- Marcus Hutter on Lex Fridman, Research Scientist at DeepMind
- Chris Olah on 80,000 Hours 1, Research Engineer at Anthropic on neural network interpretability
- Chris Olah on 80,000 Hours 2
- Paul Christiano on 80,000 Hours, previous OpenAI head of alignment
- Jan Leike on Towards Data Science
- David Chalmers on Lex Fridman
- Danny Hernandez on 80,000 Hours
- Noam Shazeer on Aarthi and Sriram's Good Time Show, one of the Transformer authors and founder of Character.ai
- Noam Shazeer on No Priors
- Felix Hill on the Embodies AI Podcast, DeepMind Research Scientist
- Ilya Sutskever on Lunar Society
Videos
Categorizations: Alignment, Safety, Capabilities, Reliability, Scalability. It was helpful for me to keep track of these categories and how they were used throughout the conversations. Occasionally even the podcast interviewers mixed them up, and of course they’re all intertwined.
A Summary of Popular Positions
- Andrej Karpathy: Vision is all you need
- Wojciech / Ilya: No one predicted how far text would get us
- David Silver: Reward is all you need
- Yann LeCun: Current methods aren’t enough. Reward isn’t enough. Need common sense from world interactions. LeCun's MO is to bash on whatever is currently most popular in deep learning research. In 2017 deep RL was most popular, so he bashed on that. He even turned out to be right, self-supervised learning did turn out to be very important. Currently self-supervised learning is most popular, so now he's bashing on that.
- Francois Chollet: We don’t have anything you need for intelligence
- Sam Altman: We seem to be in the best scenario: slow takeoff, high impact