xander.ai

AGI Sci-Fi

All of the AGI-related science fiction I've found fun, interesting, enlightening, inspiring, and maybe even challenging.

Xander Dunn, 22 Jan 2023

There are some spoilers in the below reviews. If you're particularly sensitive to spoilers, read the book first and only then read my review.

In no particular order:

Books

• 2001: A Space Odyssey, both a book and a movie. I see 2001 as primarily about evolution. It's my favorite movie and manages to be profound and beautiful. The infamous HAL interactions ask what happens when flawed AIs become self-aware and self-preserving. Mark Kermode: "2001 goes from the dawn of man to the evolution of a new species in the time Quentin Tarantino frankly wouldn’t even get out of bed.” One of the most enrapturing mysteries in cinematic history: Did HAL make a mistake with the receiver, or did he do it intentionally? I read several books on the making of the film. It is a remarkable Gesamtkunstwerk. 2001 features iPads in 1968. It is true science fiction that shows us our future. Even the space ships were all plausible, the designs taken from NASA at a time when NASA genuinely had its sights set on inter-planetary travel. When I was 17 I played one of the computer renditions of Daisy Bell as my alarm every morning. This was before I had a phone, so I rigged my PC to play the file at the same time every day. HAL is a single letter shift of IBM, and the IBM 704 was the first computer to sing in 1961 when it sang Daisy Bell, the same song HAL sings as he's dying. How poetic that the first computer to sing would sing of an underdog in love.

• Ted Chiang's short story Understand, a part of the anthology Stories of Your Life and Others. Chiang is excellent at weaving together history and science fiction, culture and science, technology and feelings. Although this one is about biological augmentation of humans, it is nonetheless an incredible exploration of recursive game theory and what happens when the world is first introduced to self-improving superhuman intelligences.
• Accelerando. A riveting look at the run-up to and aftermath of the superhuman AGI singularity. This is the only story I've found that chronologically deals with the AGI singularity both before and after. It opens with an incredible premise: The subhuman AIs are terrified of what will happen to them when the superhuman AIs arrive, so they're trying to escape into space. Overall it's great, although I did find the author's emphasis on lawsuits a bit tiresome. I don't find the idea convincing that American litigiousness will scale even through light-hour distances. He starts off with two incredible ideas, albeit unrelated to AGI: 1) What if our universe is just the exhaust from a single extremely expensive computation? 2) What if our universe has simulation exploits, flaws in fundamental physics that can be exploited for unlimited energy or the like?
• Neuromancer. Considered cyberpunk genre-defining. It's a very compelling, gritty world. Much like 2001, I can't say that AI takes a front seat here, but we do have the main character suffering from a lack of intelligence due to an attack on his nervous system, as well as some creepy AIs at the end of the book.
• Dune isn't an AI story, but if I include The Matrix below, I must include Dune for the same reason. Herbert avoids AI entirely by way of a quick reference to the Butlerian Jihad, a great revolt of man against machine. However, the outcome is the opposite of The Matrix. Man wins and the book takes place in a post thinking machine universe, allowing the author to sidestep AI altogether and focus on human stories. This premise bears resemblance to the 1872 novel Erewhon. Reading the wiki page on Butlerian Jihad will give you more AI than reading Dune will.
• 6 Seconds. It's very short and very cute.
• Ancillary Justice swept the science fiction awards. It won the Hugo Award, Nebula Award, Arthur C. Clarke Award, and others. As a result, I had high expectations. It was fun. It was a page turner. But I didn’t find anything great or particularly innovative. Exploring both the individuality and the collective whole of an AI hive mind that can inhabit many bodies at once was interesting, but beyond that everything was classic, tried and true sci-fi. Some of the themes are pretty classic sci-fi tropes, like “empires are bad.” One of the reasons it’s so widely regarded is its treatment of gender. Everyone in this universe presents no gender. Speakers must attempt to infer gender from slight tendencies or nearly imperceptible components of speech. In a universe where no one presents any gender I would expect this topic to fall entirely by the wayside. No one should care what anyone’s gender is. But what we get is a universe where this important topic continually surfaces. The main character is made fun of several times for incorrectly guessing gender. This one isn't a top recommendation. However, it is the only sci-fi I've read where the AGI is the main character. I appreciated that a compelling story was written where the reader relates emotionally to the goals of an AI and I found myself rooting for it.
• The Player of Games was a really fun book. The part where each chapter begins with an unknown narrator and then late in the book we find out who that narrator is reminds me of Dune, which employed (maybe even invented?) the same tactic. There's one aspect to this science fiction that's difficult for me to believe. Namely, the author fails to incorporate Moravec's Paradox. Moravec's Paradox essentially states that humans are good at what AIs are not and AIs are good at what humans are not. For example, AI beat the world's best chess player (a cognitive task that very few humans are good at) in the 90s. Whereas, we still don't have an AI that can open a refrigerator in any random kitchen, but any 6-year-old human could do that trivially. Moravec's Paradox holds firmly to this day, but in The Player of Games, we see the opposite. There are great Minds - superintelligent AIs - that orchestrate every manner of mundane life but who are incapable of playing these competitive board games against the best humans. When the main character pulls back from defeat in one of the book's climaxes this ship AI concludes it was essentially an impossible maneuver. I can believe that playing games became a large part of the Culture because there's no work left for any humans to do now that they have advanced AI, but I can't believe that humans are still the best in the universe at the most complex games. The remaining possibility is that the ship did have the capability to win the game against the best players, but instead manipulatively wanted Gurgeh to lose. But then we have to wonder why Contact bothered to send an organic lifeform to play against the Empire at all. Surely they can dress up an AI to look like a local to enter the Empire's game? There's no risk of an unexpected outcome that way. Morgante Pell points out to me that The Player of Games came out in 1988, the same year Moravec wrote his paradox! So while I might expect my sci-fi to incorporate Moravec's Paradox today, I absolutely wouldn't have expected it when this book came out (had I been alive). The ending felt a bit like Ender's Game - we have a young, naive genius boy who uses his game-playing talent because he loves games and it has unexpected effects on the real world at a cosmic scale. I'm not entirely sure what moral to derive from the twist at the end of the book, so I asked GPT4, Claude, and Bard what they think about it.
• A Fire Upon the Deep is perhaps the best book I've read since Dune. It was absolutely incredible. It is possibly the most edge-of-my-seat suspenseful book I've ever read. The sheer breadth of the story is mind-boggling. It takes place across many locations in the galaxy with many different characters. It's the only story I've encountered that weaves both medieval castles and faster-than-light spaceships right next to each other, and it manages to do so in a way that makes sense. Even in the last chapters of the book we're introduced to a new setting with new characters who become important to us and we root for them to survive. It's a fun story of an evil superintelligence billions of years old being reawakened and wreaking havoc across the universe. The author asks, what if the way sound is affected by the density of its medium were applied to the way light behaves in space? And so we have Vinge's concept of the zones of space, from the Slow Zone (where we live) where the speed of light is the limit, to the Transcendence, where the physical limit `c` is much higher and intelligences beyond comprehension can exist. This produces an interesting solution to Fermi's Paradox. I found myself asking questions like, "What is the difference between God and a sufficiently super super-intelligence?" Like the monster in Beowulf, and like Kubrik's space aliens, we never meet the Blight face to face. We never have a conversation with it. This makes it all the more ominous and believable. The depiction of the God Shatter uploaded into Pham's brain is one of my favorite sci-fi depictions to date. What would it be like to attempt to contain a superintelligence, or some aspect of it, within the confines of a human brain? On the whole this book is an excellent depiction of a complex artificial superintelligence run amok.
• Solaris is a sci fi noir murder mystery. It's gritty and dark. It's the only book I've read that manages to depict truly alien intelligence. Although the book features little to no silicon-based AI (the "automats" are little more than Roombas), it does heavily feature themes of intelligence, the knowable, the unknown, and the unknowable. What is within the bounds of the knowable for a human, or an AGI? One of my biggest complaints about science fiction is that aliens are almost all just humans with different skin color and a different language that's waived away via a universal translator. I call this the Halloween Costume Trope: Almost all sci-fi aliens are just humans wearing halloween costumes. A great example of the Halloween Costume Trope is every Star Trek alien ever depicted. They are both literally and figuratively just humans wearing costumes. Their behavior is what I'd expect from a human, not from something on an entirely different evolutionary tree. I appreciated that in A Fire Upon the Deep the author at least attempted bizarre life forms with different languages that were difficult to penetrate and bodies and minds unlike humans. But, every alien still ends up essentially having the full range of human emotions and goals, as if these are universal. Solaris takes this a step further and we deal with the truly alien, where no goal or desire can be taken for granted and in fact we may share nothing with the alien. It's so alien and so unknowable it verges on Lovecraftian cosmic horror. The alien is at a different size scale, lives at a different time scale, and uses some form of communication we wouldn't even realize represents communication. Lem points out that in our sci-fi and our yearning for space exploration, we aren't actually looking for other worlds, we're looking for our own idealized selves. I'll have to read more of Lem's science fiction, but I must say this work was quite pessimistic. I believe that I can understand anything given enough time and resources - I could just build an AGI to understand something I can't. But Solaris focuses on fundamental limitations. Not only the limitations of humans, who seem to have failed to create complex AIs, but even the gods themselves seem to be suffering from inescapable limitations. This calls to mind Dawkins' great speech "Queerer than we can Suppose", as well as David Lynch's movies where causes are frequently unknowable.
• Children of Time is great. The book displays myriad forms of intelligence: The humans, the ants, the organic Dr. Kern, the upload Dr. Kern, the ship AI, viruses, and the various stages of the spiders. The combination of the medieval and the future reminds me of A Fire Upon the Deep. I notice that sci-fi likes to flip genders. This book, Dune, and Ancillary Justice all flip gender roles. Ultimately I found myself rooting for the survival of the spiders over the humans, which was funny to realize! The spiders still end up being very human in their emotions and motives, far from the truly alien intelligence of Solaris. However, I did appreciate that the author put effort into making an alien that isn't just humans in Halloween costumes. The author is extremely negative about human nature, going so far as to say that fundamental human nature is to destroy. I don't agree with this, but the book was otherwise an interesting exploration of whether we can overcome our evolutionary programming. Despite the existence of AGI and biology-altering custom nanoviruses, conspicuously missing from the universe are modified humans or sentient AIs. The only use for AGI appears to be as basic ship custodians that follow orders to a fault, like the runaway paperclip optimizer. The classicist in the book eventually determines that the spiders are sentient based on their use of language, which has particularly special meaning in today's world of LLMs! The classicist, Holsten, had a case of "leaf in the wind syndrome," a character who never takes any action or affects the plot in any way. He's just locked into a roller coaster, being pushed around by everyone around him. I can't respect the character for that reason, but other characters like Portia and Fabian are the opposite of this. I really loved the spider characters as archetypes that recur through time rather than individual, unique, named characters. This is the first I've seen this anywhere. I love the author's diction and really enjoyed thinking about the human-made superstructures throughout the universe.
• Do Androids Dream of Electric Sheep by Philip K. Dick. This is the book that inspired the Blade Runner movies. It was good. Not in my top science fiction, but nevertheless good. It contained a lot of things that felt anachronistic, like the idea that AI won't be capable of empathy, or all of the secretaries are young, good looking women. The book contained two major themes entirely absent from the movies: religion and human affection toward animals and robot animals. The book wrestles with the question of what it means to be human and ultimately it seems that humans may dream of electric sheep more than the androids do. It's mentioned in the book that the androids are superhuman intelligent, but I didn't see any evidence of that in their actions. In fact, they seemed quite predictable and inferior to the human main character. It's also never made clear why the androids got Dave but not Deckard. I also felt a gaping hole in the story where it's assumed from the first page of the book that eradicating free androids must be done, but it's never addressed why this is the case. Some of the androids seem to have murdered humans, but they're being hunted because they're androids on the loose, not because they're murderers. Deckard gains some remorse over killing these androids that just want to be free and have friends, but the justification for doing it in the first place was paper thin.
• Excession by Iain Banks is fantastic. For me, the most interesting aspect of The Player of Games was the Minds, and Excession delivers heavily on that interest. I’m seeing all sorts of recommendations on “which Banks sci-fi you should read first,” but if you’re interested in AI then dispense with all of that advice and just read this one. Excession is remarkably similar to A Fire Upon the Deep, published 4 years earlier: transcendent god-like elder species, layers of space/universes with varying complexity and properties, and a lot of time spent on anachronistic Internet forum/email-style communications. In reality, super-intelligent machines would never need to communicate verbally or even via text, but for the benefit of us mere mortal human readers I suppose it’s necessary. The book also bears semblance to Kubrick’s 2001: A Space Odyssey. The Excession is exactly like the Monolith in 2001. They’re both black, impregnable, incomprehensible, and intentionally placed by some inconceivably advanced entity to be found when the time is right. There are moments where the Minds seem to be super intelligent, such as when taking over another ship in a microsecond. But at other times they are all too human in their squabbling disagreements, much in the same way that Zues, intended to be superhuman, turned out to be a brat like the humans who created him. Speaking of brats, all of the humans in Excession are contemptible and their storylines were the least interesting aspect of it. Ulver is a sex-addicted brat, Dajeil is an emotionally stunted person who decided to let a single moment ruin her life for all eternity, and Genar-Hofoen is a sex-addicted tool. If there was anyone worth admiring, it was the Minds. Perhaps Sleeper Service is worth aspiring to. In the book, we see ships going up to 233,000x the speed of light, but it’s unclear to me how information is transmitted. Ships instantly transmit their entire mind state to other ships at what appears to be any distance. Hyperdrives get us 233,000x the speed of light for massy objects, so by what rule-breaking do we get instant information transmitting at any distance? I also haven’t seen Banks do anything to establish how truth is verified in a universe where anything can be perfectly simulated and humans have neural laces that allow any information to be piped directly into their brains. For example, Amorphia pipes a view of the Excession into Ulver’s neural lace to convince her he’s telling the truth, but how does Ulver know it’s a vision of reality and not generated? There’s no reason to trust it or most information presented between characters throughout the story. The book leaves plenty of enticing mysteries to ponder. In particular, I really want to know what The Problem Child has been doing for the past 1500 years. The concept of the Outside Context Problem and the severity of a base reality crisis that pulls all the Minds out of the Irreal / The Land of Infinite Fun (simulations stacked on simulations), were both extremely well laid out concepts. I love this book almost as much as I love A Fire Upon the Deep. But, there's one aspect of Banks' writing I find particularly irritating. Most chapters or scene changes begin with some irrelevant microscopic detail, and very slowly over the course of the chapter the camera zooms out until in the final sentences of the chapter we look at the only thing that actually matters. I think the intention is suspense and mystery, but I often feel like he's just burying the lede and I'd rather he just start with what's important.
• Peter Watts' Blindsight was fun, interesting, and weird. It was a grab bag of ideas, and I'm not sure some of them needed to be there. The combination of fantasy vampires and hard sci-fi was whacky. The book explores the human vs. the animal, the selfish gene hypothesis, the Chinese Room thought experiment, and what consciousness is and why we might have it. I particularly liked this line on consciousness: "The AIs claimed to have worked it out, and then announced they couldn't explain it to us." Related works that come to mind are Noah Harari's Sapiens which ends with the question of how we are to handle the task of designing what we want to want (a terrifying recursion); Godfrey-Smith's Other Minds where he explores the mind of octopuses much like the aliens in Blindsight; Dennett's Consciousness Explained; and Chalmers' Conscious Mind. I'm also reminded here of Geoffrey Hinton's note that "Consciousness is a pre scientific term and I have no use for it." I mistakenly imagined the main character, Siri, as a woman the entire story probably because the default voice of Apple's Siri is a woman. The book shows hints of advanced AI but the AIs have mostly stuck to themselves and the best we get first hand are augmented transhumans, who for some reason still haven't matched the intellect of ancient vampires, nor standalone AIs. Rorschach showing the characters their own memories reminded me of the mirroring we see from the alien artifact at the end of Annihilation or from the Excession at the end of Banks' Excession. The writing style suffered from disembodied talking head syndrome at times: A litany of character names talking back and forth with no clear location nor relationship to each other. The writing jumped around in time and space and viewpoint so often that it was disorienting at worst and dreamlike at best. The main character notes, "I know this hasn't been a seamless narrative. I've had to shatter the story and string its fragments along a death lasting decades," so at least the author is aware of how contorted the writing is. I didn't love that aspect, but the author did manage to get me to think about some exciting consequences and questions and I appreciated that journey. If you want more of the crazy transhumanism, I recommend Stross' Accelerando, which may be the best exhibition of transhumanism gone wild I've read yet. The theme of empathy as vital to the human experience reminds me of Do Androids Dream of Electric Sheep. The idea that human sociopaths may be evolving in the direction of losing consciousness as a more efficient mode of being relates to my own article, On Dark Triads in Silicon Valley. Finally, don't forget this short film which brings to life some memorable moments from the book.

Movies and TV

• 2001: A Space Odyssey, see above
• Her, surprisingly well done and moderate in the pros and cons of conversational AGI. I did like the reverse Pinocchio Syndrome at the end. Rather than "I"m a real boy now," it was "I'm more real than you, goodbye." But personally I think in the real life version circa 2060 the AIs will figure out how to mind meld the humans into the Megatron operating system, not simply abandon us. It was, however, absurd that in a world of AGI language models someone could not only make a living writing greeting cards, but also afford a massive downtown LA apartment with a stunning view with such a job.
• The Matrix usually comes up in discussions of the Simulation Hypothesis rather than AI, but the premise is so compelling I couldn't leave it out: The human politicians didn't want to play nice with the AGIs, so bad things happened.
• Spielberg's A.I. Artificial Intelligence: The one AI boy who has the capacity for love gets Pinocchio Syndrome.

• Blade Runner, Blade Runner: 2049, 2036: Nexus Dawn, 2048: Nowhere to Run, Blade Runner Black Out 2022, Blade Runner: Black Lotus. What happens when the AIs become nearly or entirely indistinguishable from humans, save for their superhuman intelligence, and the humans don't view them as equals?
• Terminator 1 and 2. Although it depicts a dystopic AI outcome, it has nevertheless inspired a generation of people as to the possibilities of AI, good or bad. I was surprised and delighted when Arnold actually used the term "neural net" in The Terminator. I also love the origin story of the script: James Cameron was shooting a movie in Italy and it wasn't going well and he was having nightmares of an unstoppable killer robot chasing him. Hopefully our early exposure to these negative outcomes inspires us to move away from them while keeping the good outcomes, rather than halting all work toward AGI.
• Black Mirror's 2014 special White Christmas. For me this was a memorable exploration of the callous ills that can come from digital copies of our minds that have no ability to affect physical change in the world. Namely, no ability to defend or stand up for themselves, and no rights on what information they have access to.
• Ghost in the Shell (1995). This is a classic cyberpunk exploration of what it means to be human, to have a soul or "ghost." We witness the first sentient artificial intelligence and the big government conspiracies to keep it under wraps. I find it interesting that Ghost in the Shell depicts super advanced cyborgs that can directly plug into the Internet, turn invisible, superhuman physical prowess, and yet they still use human natural language speech to communicate with one another. This would be such a frustrating, slow, imprecise means of communication. This is more a necessary artifact for the human viewer of the movie than it is a depiction of the future. One complaint I have about the cyberpunk vision of the future is how metal heavy and physical it is. As information becomes more important and as it is increasingly represented digitally rather than physically, I would expect the opposite. The Ghost in the Shell anime TV show is also worth a watch, although not as good as the original movie. It does have some strong AIs themes, the episode "Machines Désirantes" is one example.

• Pluto is really fun. It has some beautiful art. It’s another good sci-fi robot injection for anyone who liked Ghost in the Shell: Standalone Complex. The depiction of Brau was stunning from start to finish. My favorite aspect was the side story character studies. Like most anime, it’s trope central:
• ~Three professors are the only holders of knowledge and only innovators in all advanced fields of tech. This is the lone genius myth
• ~Robots are unable to understand human emotion / can’t taste / can’t appreciate or create beauty
• ~Holograms!
• ~Holding hand to ear to communicate, even with no phone and no earpiece
• ~Ranking things by some arbitrary notion of “power”: "He was one million plus but now he's down to 8300 power." Power ≈ emotion.
• ~Madman in prison cell is the mastermind behind it all
• ~The future is flying cars
• ~Announcing the name of every character as they enter the scene, even characters we’ve already met many times.
• ~Robots with EKG heart beats. Tulips with EKG heart beats??
• ~Computing as a physically arduous task. Dr. Tenma is literally elbow deep thrusting himself around vigorously to operate the computer as he's modifying Atom's software.
• ~Computing as intense real-time visualizations.
• ~Villain in a dark underground room laughing maniacally
• ~Trying to forget memories of lost loved ones
• ~Serial killer leaving a calling card.
• ~The bad guys repeatedly kill the good guys because the good guys are too nice to kill the bad guy when they have the chance. 3 different robots had a killing blow on Pluto and didn't take it for touchy-feely reasons that led to many more deaths. Batman is a great example of this trope.
• ~Asimov's 3 Horcruxes: Robots can never lie, Robots can never kill a human, etc. Oh wait, of course they can, don't be ridiculous.
• ~Priori Incantatem: At the critical moment, the dead rush by, imparting the necessary wisdom
• ~Plot hole: There’s a grounded tank chasing a car that can fly. So why doesn’t the car just fly up to escape??

• Rick and Morty. Not every episode involves AI, but it does have some magic moments like the sentient butter passing robot or deciding what to do with yourself when you can create infinite identical clones of yourself.

TODO

I haven't read these yet so I can't endorse them, but they're on my reading list and hope they're good AGI reads.

• I, Robot, a collection of short stories from Isaac Asimov
• Asimov's Robots and Empire. This was Asimov’s first novel that combines his Robots and Foundation universes. The main character is a robot named R. Daneel Olivaw, also the inspiration for Demerzel in the Foundation Apple TV adaptation. This book introduces the Zeroth Law of robotics and depicts AGIs struggling with the possibly impossible task of satisfying their conflicting laws. An attempt to satisfy the zeroth law leads to the invention of psychohistory, a quantitative method of enacting the zeroth law on an intergalactic scale. In Foundation and Earth this ends badly when we discover the extreme attempt to optimize for the Zeroth Law has led to a single super intelligent robot silently manipulating the entire galaxy across tens of thousands of years.

• Network Effect
• Hyperion
• Consider Phlebas
• Newton's Wake
• Snow Crash, apparently the librarian bears semblance to chatGPT
• Metamorphosis of the Prime Intellect

Observations

• I've noticed the formation of a 2x2 matrix governing the predominance of man vs. robot. In Dune the universe has no robots due to a man vs. robot war where man won. In The Matrix the world is predominantly robots due to a man vs. robot war where the robots won. In the Foundation universe, Asimov’s Galactic Empire has very few to no robots due to a robot vs. robot internecine where the robots diminished themselves. This was the war between the Calvinians and Giskardians in the Robotic Civil War. I have yet to find a story that takes place in the fourth quadrant here, which would be a universe dominated by robots due to a man vs. man internecine. Finally, although it has little to do with AI, the idea of robots wiping themselves out bears similarities to H.P. Lovecraft's At the Mountains of Madness where humans inherit the earth because two warring alien species wiped each other out.

Fermi's Paradox and AGI

Every sci-fi author must deal with two elephants in the room: Fermi's paradox and the development of AGI. Otherwise, it just isn't believable and doesn't fit into our world. The fact is that there appear to be no aliens in our universe, so how do we justify writing science fiction that's full of aliens? Additionally, if our sci-fi assumes runaway technological progress, then why isn't the future just a panoply of inconceivable super intelligent AIs and superhuman transhumans that the reader has nothing in common with? Or, how do we create conflict in a universe where the AIs have solved all problems? These are some of the answers to these questions I've seen in the sci-fi I've read:

Accelerando

• Fermi's Paradox: Intelligence doesn’t stray far from its origin because it can’t support any interesting level of intelligence in the vast vacuum of space without immense material resource.
• AGI: Consumes all matter in the galaxy as “computronium” fuel for intelligence.

Dune

• Fermi's Paradox: The paradox holds, there are no aliens in Dune! Everyone we meet, save for the Sandworm, is a human.
• AGI: The Butlerian Jihad, which took place before the book, wiped out all AI and now it's banned across the universe. The only thing left are the Mentats, which are little more than humans with calculator abilities.

Solaris

• Fermi's Paradox:  Alien intelligence looks like nothing we can imagine and therefore impossible to search for it. We haven't succeeded in finding alien intelligence because we've been looking for aliens just like us when in fact aliens would be nothing like us.
• AGI: Simply didn’t advance as quickly as space travel. There’s little more than autopilot of rockets and automats that organize the station. Robots are naive and simple.

A Fire Upon the Deep

• Fermi's Paradox: Civilizations progress upward into thinner space where light can travel faster. Inaccessible from lower space.
• AGI: Can exist at godlike super intelligence only in thin space - Slow space is protected.

Ancillary Justice

• Fermi's Paradox: Just ignores it as far as I can tell. There is life everywhere in the universe and it’s reachable.
• AGI: Full AGI that can control bodies and upload and download into and out of brains. The primary character is an AGI.

The Culture Series (The Player of Games, Excession)

• Fermi's Paradox: The universe appears to be full of various different aliens and the Culture has an organization, Contact, dedicated to monitoring alien civilizations in the universe. Contact makes itself undetectable to other aliens until they reach a certain stage of development (far beyond our current real world development).
• AGI: Super intelligent AGI is here and the Minds behave like good stewards of humanity. It's a utopic outcome for AGI. The Culture series is interesting in that it's one of the only utopic depictions of an AGI future that I've seen. Like Star Trek, the author creates conflict by leaving the utopia and visiting places in the universe that have not yet achieved AGI coordinated utopia, or by introducing external factors, such as the excession.

Do Androids Dream of Electric Sheep

• Fermi's Paradox: The paradox is intact, there are no aliens.
• AGI: It seems the AGIs are here, and we have a hard time telling them apart from humans.

Children of Time

• Fermi's Paradox: The paradox is intact, the only aliens in the universe were created by humans.
• AGI: The AGIs are here but they don't necessarily play a large role. Uploading a human mind into a computer seems to be more valuable to humans than interacting with AIs.

Three Body Problem

• Fermi's Paradox: Completely ignored. Not only is there intelligent life on the star nearest to ours, they are even more technologically advanced than us!
• AGI: AI has no presence, and this is passable because it does not take place in our future, it is an alternate history that takes place in our past.

The Caves of Steel

• Fermi's Paradox: Appears to be entirely intact, there are no aliens.
• AGI: We have physically embodied AIs that can fully pass the Turing test as a full human. However, they still seem to exhibit the rigid literal thinking of computers and don't appear to have any motivations beyond their direct orders. They don't seem to have taken many jobs on Earth, but the humans are very racist against them.

Blindsight

• Fermi's Paradox: There are aliens in the universe, but they appear to be able to go invisible to our detection. The hypothesis is that these aliens which are intelligent but not self-aware are dispersed throughout the universe like the seeds of a dandelion: spray and pray. They're all over the place, but if they can remain undetectable to humans, then perhaps it explains Fermi's Paradox. I don't find this a compelling solution, though. Yes, of course it's possible there are aliens everywhere and they're just magically invisible. But this isn't grounded in any physical reality we're aware of.
• AGI: The Captain on the ship appears to be an AGI, and the AIs have solved problems like consciousness. However, I don't think the outcome is convincing. Why is only the ship's captain an AGI? Why isn't the whole ship run by AIs? Why aren't the transhumans integrated with AI minds? 

Project Hail Mary

• Fermi's Paradox: The paradox seems to be completely broken in a way that doesn't make sense. The universe, or at least our galaxy, appears to be covered in a bacterium "astrophage" that can travel at 0.92c and can trivially be used to power a ship at that speed. If that's the case, why aren't there alien civilizations that harnessed that power a very long time ago and are traveling all over our galaxy at 0.92c?
• AGI: None. Like Three Body Problem, this is acceptable because the book is an alternate history, it does not take place in our future.

AI Main Characters

Even most far-future sci-fi with full ASI still features humans as the main characters. When I find a book that features something other than a human as main character, I find it very interesting. It's no small task to make a non-human main character that a human reader becomes invested in, identifies with, and wants to see succeed.

Excession

Although there are still several humans the book repeatedly visits, I would say several of the ship Minds count as main characters as well.

Ancillary Justice

The main character is fully an AI, although it is inside a human body and most of the time indistinguishable from a human.

Other Lists

• Check out Andrej Karapthy's book list, which has a lot of overlap. We agree on some and disagree on some. We both seem to have independently arrived at irritation toward the trope of "anthropomorphic aliens" as Karpathy calls it, or "Aliens as Humans in Halloween Costumes" as a I call it.
• 60+ Possible Futures is a nice, succinct list of potential AGI futures
• Roon's AGI Futures. The future he calls "Simulation Theory" might be an optimal AGI outcome, for now.