xander.ai

AGI Non-Fiction

Some of the AGI-related non-fiction I've found interesting and useful.

Xander Dunn, 30 Jan 2023

In no particular order.

Books

• Human Compatible is a pretty good exploration of the risks of AGI and a call to arms to research safety and alignment. At times the author can be sensationalist, such as in the killbot video he disseminated, but overall it's a well-reasoned and sufficiently balanced discussion of both the great positives and potential negatives of superhuman AGI. It was mostly a refresher on things I'd already thought of, the audience that will benefit the most from it are people who are not already familiar with the state of the art in AI.
• Judea Pearl's The Book of Why. This book is not about deep learning and can be skipped if that's your focus. It's an interesting account from a deep learning nay-sayer who thinks we will have to take altogether different approaches to arrive at models that understand causality. Some of the causation paradoxes are interesting, and the earlier chapters sharpened my thinking on Bayesian inference. It was an easy listen while I was hiking.
• The Technological Singularity is written by Murray Shanahan, a professor of AI for robotics, so it's worth considering more than most writings on this topic. I think it would've been very prescient in 2015, but it's both written for an audience that isn't well versed in AI and it needs an update given what we know today. The first 6 chapters (on audiobook) are dedicated to convincing the reader that AGI is possible. This whole discussion is much less necessary today than it was in 2015. It asks some of the standard questions: Could/should AIs be conscious? How do we imbue the AIs with empathy for humans? Should the AIs have rights, and what rights? Would a self-improving AI lead to a sudden intelligence explosion? There's a fun discussion of the "neural substitution argument," which first appeared in Hans Moravec's 1988 book Mind Children. Personally I find these philosophical Sorites paradox discussions unnecessary. I'm a functionalist, so it seems obvious to me that consciousness like mine can exist on a digital substrate. It does correctly point out that being superhuman at all tasks does not require having any form of sentience or consciousness. It makes the important distinction that intelligence need not be correlated with capacity for suffering. The book makes some of the standard mistakes everyone made in 2015. It defines passing the Turing Test, translation, programming, and performing law as AI-Complete, meaning solving these problems would require fully solving general intelligence. Today we see that's likely not the case. The Turing Test is largely solved for most people, AIs are better than humans at translation, and AIs are rapidly encroaching on skilled programming and legal work. Despite this, no one points to chatGPT and says strong AI has been achieved. It's still nowhere near the ability to open a refrigerator door with human-level acuity. The author sees the primary difference between the Industrial Revolution and AI as the former creating as many jobs as it replaced, while the latter may simply replace jobs. I think it's still a good read for someone who isn't familiar with the space, but I would really love to hear the author's thoughts in 2024 for an audience that is aware of what's going on.

Textbooks

• Sutton & Barto's Reinforcement Learning is a great introduction to reinforcement learning. It's accessible and well-written. I read the first edition cover to cover in 2015. The second edition is out now and I suspect it's just as good. The only criticism I've heard of it is that it makes RL seem easier than it is, which may be the case. Andy Barto accepted a demotion from professor to postdoc to move to a different university to start working on RL. The planning and Monte Carlo Tree Search sections in this book are highly relevant throughout deep learning.
• MacKay's Information Theory, Inference, and Learning Algorithms. I have a personal strong interest in Information Theory, and this is the only AI textbook I've found that uses information-theoretic notions throughout its discussion of AI. Other books, like Goodfellow's Deep Learning textbook, have one tiny subsection of a chapter devoted to it. It's all but swept under the rug. I haven't read MacKay's book cover to cover but I have read certain chapters and find it accessible.
• Theory of Knowledge, Second Edition. This was recommended by Marcus Hutter, the information theorist at DeepMind. Specifically the second edition, as he says "they took out all the fun" in the newer editions. This is epistemology, or how we know what we know. This book is really fun and I would've loved it as a high schooler.

Other

• You and Your Research by Richard Hamming is a popular resource for researchers in any field, including in AI. He also gave a talk on this here.
• The Bitter Lesson from Richard Sutton is a seminal insight in the field.
• The Future of Work and Death was more exciting in its premise than its delivery. I watched it because it had interviews with Murray Shanahan. It does have a bit of doomer to it. From the opening it has ominous deep thumping bass to evoke edge of your seat incoming disaster. It also manages a graph with no Y axis. All of the people they interviewed massively failed to appreciate Moravec's Paradox. They're all worried about fast food workers and factory workers losing their jobs first, when what we're seeing is the opposite: AI becomes super human at communication, coding, and art long before it can even open an arbitrary refrigerator door at human-level capability, let alone replace complex physical labor. Spent too much time talking about 3D printing and not enough time talking about AI that can replace knowledge workers. The ditch digger outlasting the AI research engineer was not in the cards in 2015. In 2015 I was working at an AI for robotics company. I too didn't appreciate Moravec's Paradox sufficiently, but we're over that hump and it's time for a new documentary.
• Moravec on Moravec's Paradox. Moravec's Paradox, in my opinion, stands right next to the Bitter Lesson as one of the most important and most persistent realizations in AI research so far.

TODO

I haven't read these yet so I can't endorse them, but they're on my reading list, and hope they're good reads.

• The Structure of Scientific Revolutions
• The Conscious Mind by David Chalmers
• Consciousness Explained by Daniel Dennett
• A Brief History of Artificial Intelligence by Michael Wooldridge