This Harvard Professor Says AI Alignment Will BACKFIRE - Dr. Stephen Casper

Stephen Casper, PhD, is an incoming tenure-track professor of public policy at the Harvard Kennedy School.

Prof. Casper is relatively unworried about the threat of human extinction from rogue superintelligence, but he is concerned about a handful of companies wielding extremely powerful AI and enfeebling the public. That’s why he’s an AI governance hawk who advocates for taxes and regulation to keep frontier labs in check.

We agree that slowing down AI development would make the world safer. But you know his position is unique when he says he’d prefer to have less research on AI alignment!

Timestamps

00:00:00 — Cold Open

00:00:50 — Introducing Stephen “Cas” Casper

00:04:21 — Cas’s MIT Research: Model Tampering Attacks

00:07:14 — What’s Your P(Doom)?™

00:08:34 — Why a 5–10% P(Doom) Feels Patronizing

00:13:07 — Crux: “The Intelligence Ceiling Might Not Be That High”

00:17:18 — Power Structures vs. the Shortcut-Finding AI

00:26:03 — Thought Experiment: A Data Center From the Year 2126

00:28:25 — Is a Data Center as Dangerous as a Nuke?

00:33:41 — Cas’s Mainline Scenario: Slow-Burn Gradual Disempowerment

00:40:49 — A Multipolar Powder Keg: 10 Companies, Thousands of Open Models

00:43:48 — Boarding the Doom Train at a Later Station

00:45:49 — Why Cas Is an “Anti-Timelines” Person

00:52:26 — Sycophancy, MechaHitler & Nudification: Foreseeable Failures

00:57:18 — Crux: Does Humanity Get Retries?

01:02:39 — The Scariest Superintelligent Optimizers Are Companies

01:06:08 — Lessons From the Deepfake Ecosystem: DALL-E 2 vs. Stable Diffusion

01:12:42 — The Case for a Pause Treaty

01:16:35 — Whac-A-Mole Forever: What Winning Looks Like

01:20:51 — Enshittification and the 84% Who Don’t Use AI

01:30:30 — Alignment Research as Safety Washing

01:34:25 — The Jevons Paradox of AI Safety

01:36:48 — Cas Would Press the Button to Halt Superalignment

01:39:21 — Wrap-Up: The Unsexy Path to Lowering P(Doom)

Links

Cas’s links

* Cas’s website

* Cas on X

* Cas’s Google Scholar

* Cas’s MATS stream

Things referenced

* Superintelligence by Nick Bostrom

* Gradual Disempowerment (Duvenaud et al.)

* If Anyone Builds It, Everyone Dies

* AI 2027

* UK AI Security Institute

* Center for Human-Compatible AI (Berkeley)

* Internet Watch Foundation

Doom Debates episodes mentioned

* Mike Israetel Returns — AI’s Gonna Kill Everyone vs. AI Will Make Everything Awesome

* Top AI Professor Has 85% P(Doom) — David Duvenaud

* He Leads a Top AI Research Program, But He’d Hit the PAUSE Button — Kevin Zhu

* Alignment is EASY and Roko’s Basilisk is GOOD?! — Roko Mijic

* Andrew Critch vs. Liron Shapira: Will AI Extinction Be Fast Or Slow?

* AI Could Give Humans MORE Control — Ozzie Gooen

* Steven Byrnes Part 1

Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.

Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates, or to really take things to the next level: Donate 🙏

Get full access to Doom Debates at lironshapira.substack.com/subscribe