Debating MIRI President Nate Soares About AI Doom

In this episode of Seeing Clearly, MIRI President and co-author of If Everyone Builds It, Everyone Dies Nate Soares and I discuss existential risk from artificial super intelligence. We first run through the case for AI doom before I push back on some of his assumptions.

Nate Soares is the President of the Machine Intelligence Research Institute (MIRI) and co-author (with Eliezer Yudkowsky) of If Anyone Builds It, Everyone Dies

Order Nate’s Book: https://amzn.to/4wpQXve

0:00 — Highlights

0:40 — Is super intelligence inevitable?

2:07 — The default outcome

3:09 — AI Scepticism vs. Takeoff

11:01 — What might ASI want?

17:37 — Why would ASI kill us all?

18:20 — Would ASI change its own goals?

23:21 — Would ideal humans values converge?

24:53 — The fundamental assumption in Rationalism

35:47 — Does moral realism imply alignment?

56:42 — p(moral convergence)

1:00:35 — Will ASI be an optimizer?

1:04:13 — Emotionally dealing with potential apocalypse

Get full access to Seeing Clearly at cxgonzalez.substack.com/subscribe