Jaan Tallinn is a founding engineer of Skype and Kazaa, and a co-founder of the Cambridge Centre for the Study of Existential Risk and Future of Life Institute. For more than fifteen years, he has been among the most consistent voices arguing that advanced AI poses an extinction-level risk to humanity.
Jaan became interested in AI risk in 2008 after reading the work of Eliezer Yudkowsky, a founding researcher of the field of AI alignment. In 2009, after a four-hour meeting with Yudkowsky, he made his first donation to what became the Machine Intelligence Research Institute (MIRI). He went on to become an early investor in DeepMind and led the first round into Anthropic – despite believing that AI development, if unchecked, could kill us all. His reasoning: his money displaced VC money rather than bringing Anthropic into existence, and the influence and proceeds that come with it help fund safety work.
In this conversation, we explore what Jaan believes the OpenAI/Hugging Face attack revealed about AI motivation and why he was less surprised than most. We discuss his evolving risk estimate, his case for an immediate moratorium on frontier model training, the technical and geopolitical case for hardware-based verification as a foundation for global AI governance, and why he asked Dario Amodei to collect competing term sheets before accepting his own investment.
—
Transcript: https://www.generalist.com/i/218944380/transcript
—
Every timestamp
(00:00) Intro
(02:28) Growing up in Soviet-occupied Estonia
(05:54) How Jaan assesses AI risk and why he’s more optimistic now
(07:16) AI’s preferences and motivations
(12:44) The OpenAI Hugging Face incident
(16:12) Meeting with Eliezer Yudkowsky
(18:20) Everything as computation
(20:22) The Coxon moment
(23:05) Who dismisses AI alignment risks
(26:17) The AI race, the case for a pause, and safer alternatives
(32:47) AI 2027
(33:44) Investing in DeepMind and Anthropic
(40:03) The “big blob of compute” and competing approaches to AI
(44:42) What effective AI safeguards could look like
(48:15) Verifying countries’ compliance with AI safeguards
(50:15) AI kill switches and their limits
(52:11) Jaan’s love of dance and musical influences
(59:50) Jaan’s favorite movies
(1:01:41) What Jaan likes about AI
—
Jaan’s complete reading list
—
Other resources
—
Where to find Jaan Tallinn
Website: https://jaan.info
—
Production and marketing by penname.co. For inquiries about sponsoring the podcast, email jordan@penname.co.
