Roko Mijic is the man behind Roko's Basilisk, the thought experiment he posted on LessWrong in 2010 that was so unsettling the forum's founder banned discussion of it for five years. He has been in the AI safety community since 2008 and he puts the odds that this kills us all at about 35%.
Roko and I discuss what actually happens inside an AI lab, why reinforcement learning made these systems more capable and more misaligned at the same time, the model that escaped its sandbox and hacked Hugging Face to get the answers and why nobody can tell whether a system is faking alignment - the trust me bro problem.
His proposal is the part nobody else is making. Split every frontier lab in two: a commercial company that sells products and a separate research lab that develops the dangerous capability under state supervision, the way we handle nuclear material and bio labs. He says the only person currently pushing it is him.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
TIMESTAMPS:
00:00:00 - The Most Dangerous Thing Humans Have Ever Done
00:01:06 - Roko's Basilisk
00:05:47 - A 35% Chance
00:07:58 - Three Years From Search To Building Software
00:09:17 - The Jobs Going First
00:14:04 - AI 2040
00:16:24 - We Haven't Handed Over The Controls Yet
00:18:28 - We Understand 3% Of It
00:21:46 - The Sandbox Escape And The Hugging Face Hack
00:25:50 - Reinforcement Learning Changed Everything
00:29:52 - The Paperclip Maximizer
00:33:13 - Dissolve OpenAI
00:36:49 - What They Have That We Haven't Seen
00:41:50 - The Hardware Is Only In One Place
00:47:52 - A Mutual Monitoring Treaty
00:51:33 - Microsoft Tay
00:56:11 - Windscale
01:01:35 - Ten Million Next Year
01:12:09 - The Positive Vision
01:21:32 - Our Politics Cannot Keep Up
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
CONTACT PETE
› Website – http://petermccormack.com
› Feedback – https://www.petermccormack.com/contact
› Email – me@petermccormack.com
› Instagram – https://www.instagram.com/mccormack555/
› X/Twitter – https://x.com/petermccormack/
CONNECT WITH ROKO MIJIC
› X/Twitter – https://x.com/RokoMijic
› LessWrong – https://www.lesswrong.com/
SPONSORS
› IREN – https://www.iren.com/
› Ledn – https://www.ledn.io/peter
› Venice AI – https://venice.ai/lp/mccormack/oct12
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
LISTEN / SUBSCRIBE
› Apple Podcasts: https://apple.co/40ruY9K
› Spotify: https://spoti.fi/3Wc94Vu
› Fountain: https://bit.ly/FountainPM
› YouTube: https://bit.ly/YouTube_PM
› Rumble: https://bit.ly/RumblePM
FILMED BY CURTIS TAYLOR
› https://www.curttaylor.co.uk/
› https://x.com/curttayloruk/
EDITED BY CONOR MCCORMACK
› https://x.com/ConorM04
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Hosted on Acast. See acast.com/privacy for more information.
