Garrison Lovely is an award-winning journalist and author of the highly anticipated book, Obsolete: The AI Industry’s Trillion-Dollar Race to Replace Us and How to Stop It. I give it the Doom Debates stamp of approval!
While we’re mostly on the same page about the AI arms race, Garrison and I clash over Eliezer Yudkowsky’s role. He blames Eliezer for kicking off the superintelligence race and refers to his Coherent Extrapolated Volition (CEV) vision as a “coup on humanity.” But I say it was actually Yud’s attempt to keep humanity in charge.
Can Garrison change my mind?
Watch on YouTube: https://www.youtube.com/watch?v=VZ94J4DBRkw
Timestamps
00:00:00 — Cold Open
00:01:05 — Introducing Garrison Lovely
00:04:23 — “Freeze AI” vs. “Pause AI”
00:06:17 — Garrison’s Background at McKinsey
00:10:31 — Should ICE Be Abolished?
00:17:07 — How Garrison Got on the AI Beat
00:22:16 — What’s Your P(Doom)?™
00:23:29 — Focusing on Extinction Is a Distraction
00:25:46 — Is Stopping AI an Easy Coordination Problem?
00:29:02 — Society’s Immune Response to AI
00:33:54 — Garrison’s P(Extinction) by 2050
00:37:09 — Did Eliezer Yudkowsky Start the AI Race?
00:45:56 — Freeze AGI Until the Public Opts In
00:50:32 — Is Coherent Extrapolated Volition a Coup?
01:02:17 — Democracy, Informed Consent, and the Singularity
01:11:39 — Should Eliezer Have Raised the Alarm Sooner?
01:16:45 — “Classic” AI Safety Burned Its Credibility
01:19:18 — Garrison’s Proposal to Stop the Race
01:21:35 — Will China Agree to a Freeze?
01:25:14 — Let’s Stop the “Obsoleting Project”
01:26:00 — Wrap-Up
Links
GARRISON LOVELY
“Obsolete: The AI Industry’s Trillion-Dollar Race to Replace Us—and How to Stop It” — Garrison’s book
Obsolete — Garrison’s Substack
Garrison Lovely on X
Organize Against the Machine — Garrison’s new podcast with labor organizer Cassie Pritchard
“Can Humanity Survive AI?” — Garrison’s Jacobin cover story
“McKinsey & Company: Capital’s Willing Executioners” — Garrison’s Current Affairs essay
REFERENCED IN THE EPISODE
Open Borders: The Science and Ethics of Immigration — Bryan Caplan
“The AI Revolution: Our Immortality or Extinction” — Wait But Why
Superintelligence: Paths, Dangers, Strategies — Nick Bostrom
“AI 2040: Plan A” — AI Futures Project
“An easy coordination problem?” — Katja Grace
The Hugging Face incident and the road ahead — OpenAI
“Pause AI for Humanity’s Sake” — Peggy Noonan
“Pause AI Development NOW” — Bernie Sanders
If Anyone Builds It, Everyone Dies — Eliezer Yudkowsky & Nate Soares
“Staring into the Singularity” — Yudkowsky’s 1996 manifesto (archived)
“Coherent Extrapolated Volition” — Eliezer Yudkowsky, 2004 (PDF)
“Coherent Extrapolated Volition: A Meta-Level Approach to Machine Ethics” — Nick Tarleton, 2010 (PDF)
Solomonoff induction — Wikipedia
“Personal statement on joining the OpenAI board” — Paul Christiano (4% over the next year)
“Darwin among the Machines” — Samuel Butler, 1863
PauseAI
PAST DOOM DEBATES EPISODES MENTIONED
Are We A Circular Firing Squad? — with Holly Elmore, Executive Director of PauseAI US
Debate with Robert Wright: Will Humanity Pass the “God Test”?
Robin Hanson vs. Liron Shapira: Is Near-Term Extinction From AGI Plausible?
---
Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.
Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏
Get full access to Doom Debates at lironshapira.substack.com/subscribe
