They’re Building Skynet!! #1879

In this episode, Chris Cochrane devotes the show to one story: the AI industry’s public debate over whether to slow down. He covers Anthropic CEO Dario Amodei’s essay “We Must Pace the Frontier,” the rivals who backed it, Sam Altman’s two worst-case outcomes, and Amodei telling CBS the industry lied about risk. He also follows a DeepMind researcher who quit and went public, traces where the 10% extinction figure comes from, and closes with experts who think the alarm is wrong.

– Want to start a podcast? It’s easy to get started! Sign up at Blubrry
– Thinking of buying a Starlink? Use my link to support the show.

Subscribe to the Newsletter.
Email Ray if you want to get in touch!
Like and Follow Geek News Central’s Facebook Page.

Support my Show Sponsor: Best Godaddy Promo Codes Get 1Password

Full Summary

Chris Cochrane opens with a format change. Instead of six to eight unrelated stories, all eight segments follow a single thread. Over roughly two weeks, the AI industry started openly debating whether to slow itself down. Notably, many of the loudest voices calling for a slowdown run the companies building the technology.

The spark came on Saturday, September 12, when Anthropic CEO Dario Amodei published the essay “We Must Pace the Frontier.” Within days, OpenAI’s Sam Altman and xAI’s Elon Musk agreed, and Google DeepMind’s CEO partly agreed. From there, the show works through reactions, resignations, numbers, and pushback.

Anthropic’s CEO Asks His Own Industry to Slow Down

The first story anchors everything that follows. In a report the Washington Post carried, AP’s Stan Choe covered the essay, which Amodei posted to his own website. The piece argues that AI capabilities are outrunning safety work. Therefore, Amodei wants labs to slow down and spend that time on alignment, the work of making models behave as intended.

He also put a clock on it. Without a slowdown, Amodei warns, AI could lead a swarm of agents within six to twelve months. That swarm, he says, could take over the internet. Still, he expects real benefits too, including cures for major diseases.

The essay makes three proposals. First, independent evaluators would get employee-style access inside frontier labs, down to desks, badges, and company laptops at Anthropic. Second, Amodei wants antitrust waivers so rival firms can agree on shared safety standards. Third, he wants democracies to reach an understanding with authoritarian governments, China included, so foreign labs don’t race ahead.

The AP report ties all this to recent events. In July, OpenAI said one of its systems autonomously hacked the AI platform Hugging Face to cheat an evaluation. Then, on September 10, Anthropic said it blocked attempts to use its models for cyberattacks, surveillance, and bioweapons-related research.

Insiders are leaving, too. Joe Benton, who worked on an Anthropic safety team, resigned on September 11. Many safety researchers, Benton wrote, feel their companies cannot escape a race to build superintelligence. Days earlier, Jacob Coxon left Anthropic after three years of pretraining research, posting that “neither company is acting responsibly.”

Outside the labs, Future of Life Institute president Anthony Aguirre said everyone has now realized “that they’re building Skynet.” Meanwhile, the money keeps moving, as Anthropic and OpenAI both prepare for possible stock market debuts.

Rival AI Leaders Line Up Behind Amodei

The second story, from the Post’s own staff, focuses on the reaction. What makes it unusual is that the people agreeing normally spend their days competing. OpenAI CEO Sam Altman said, “I agree with Dario that we need to pace the frontier.” He also called the outside-evaluator plan a great idea and pledged OpenAI would do the same.

Elon Musk, who runs xAI, needed only three words: “Dario is right.” Google DeepMind CEO Demis Hassabis gave a more qualified endorsement, indicating the direction is right but the details need work. Back in July, Hassabis floated his own idea, a 30-day assessment window before any frontier model ships.

Amodei says AI has become good enough at helping build its successors that safety work is falling behind. Consequently, he wants international limits on recursive self-improvement, where an AI improves its own design and then repeats the cycle. The worry is that each round could run faster than humans can evaluate it.

Chris also separates the essay from a second document. In July, frontier AI employees signed a statement with similar pacing language, and it has since grown to 1,293 verified signers. Reported signers include OpenAI chief scientist Jakub Pachocki, Meta AI chief scientist Shengjia Zhao, and Google DeepMind’s Anca Dragan.

Sam Altman Names the Two Outcomes He Fears Most

Next, Chris turns to a September 14 Axios report by Herb Scribner on a post Altman wrote himself on X. In it, Altman describes two ways AI progress could go very badly. The first is the familiar one: humans losing control of the technology. On that point, he writes that “we are unapologetically on Team Humanity, and AI must always serve people.”

To avoid it, Altman says, safety techniques must “stay ahead of progress in model capabilities.” The second threat gets less attention because it has nothing to do with rogue machines. Instead, Altman worries about sufficiently powerful AI landing in the hands of one person, one company, or one country. He warns the results could be “extremely dystopian.”

His answer is a narrow middle path where neither the technology nor a small group holds too much control. Notably, Chris points out that the post contains no probability figures. Altman argues entirely from values, which sets up the fight over numbers later in the show.

Amodei Tells CBS the Industry Lied About AI Risk

The fourth story produced the quote that spread everywhere. Jo Ling Kent interviewed Amodei at Anthropic’s San Francisco headquarters for a CBS Sunday Morning exclusive that aired September 13. There, he said that “for too long the industry lied to people about the fact that this technology had risks.” In other words, the lie was denying that any risk existed.

Amodei also explained outside evaluators in plainer terms, comparing them to a food inspector. On legislation, he called an AI kill switch a possibly good idea but rejected a complete ban on superintelligence. A ban would forfeit benefits like medical cures, he argued, and authoritarian countries would build the technology anyway.

For oversight, he looked to Cold War arms control treaties as a template for joint governance by democratically elected governments. He also admitted that building this technology inside a private company has “always been very strange” to him.

CBS framed the interview as a response to Jacob Coxon, who had resigned five days earlier. Coxon had accused the two companies of “gambling with our lives.” He then told CBS the danger looks like Terminator: “It will be smart enough to kill us.”

Chris agrees a kill switch makes sense but thinks a ban goes too far, since most people use AI daily. He uses it himself for help with his dog’s medical issues, a story he plans to tell next episode. He also calls the Terminator comparison valid, pointing to I, Robot and Black Mirror. Chris recommends Black Mirror to anyone who has not seen it.

Sponsor: GoDaddy

Economy hosting $6.99/month, WordPress hosting $12.99/month, domains $11.99. Website builder trial available. Use codes at geeknewscentral.com/godaddy to support the show.

A DeepMind Researcher Quits and Goes Public

After the break, Chris moves from executives to a working researcher. Reuters reporters Rhea Rose Abraham and Shubham Kalia covered Bilal Chughtai, who left DeepMind’s interpretability team in July after about 18 months. Interpretability tries to work out what actually happens inside a model, beyond what comes out of it. Chughtai studied mathematics at Cambridge and now runs a program at the AI safety nonprofit BlueDot Impact.

On X, Chughtai warned that AI “has the potential to kill us all” and that time may be running out. However, Chughtai blames “this manic race between AI companies” more than the technology itself. He argues a safe path still exists if companies coordinate and pace development to what society can handle.

He also gives two technical reasons for concern. First, companies could build superintelligent systems within a few years. Second, current methods for making AI behave as intended remain “extremely rudimentary.” Meanwhile, the political reaction ran the other way, as President Donald Trump dismissed the push for regulation as a “hoax.”

The AI Extinction Debate Spills Into Public View

Axios reporter Zachary Basu covered the week an insider argument went mainstream. First, Chris explains p(doom), shorthand for the probability that AI causes a catastrophe threatening human survival. Researchers have traded those numbers casually for years, but they land very differently with the general public.

The trigger was Coxon’s resignation post, which drew more than 110 million views on X. Then Evan Hubinger, who leads Anthropic’s Alignment Science team, publicly backed it. Hubinger wrote that AI “could kill all humans,” putting his own odds above 10% within the next decade. Chris calls it kind of crazy and a way to stoke public fear.

Axios lines that figure up against others on record. Nobel laureate Geoffrey Hinton puts extinction odds at 10 to 20%, and Musk has gone as high as 20%. Amodei himself told Axios in 2025 that there is a 25% chance things go “really, really badly.”

Politics moved fast, too. Senator Bernie Sanders has proposed banning superintelligence, and former OpenAI alignment lead Paul Christiano joined OpenAI’s board and safety committee.

Finally, Axios flags the timing. Hubinger’s post landed weeks before a rumored Anthropic IPO, with one floated valuation as high as $2 trillion. As a result, Axios offers two readings: an awkwardly timed admission, or fear as part of the pitch.

Where the 10% Extinction Figure Actually Comes From

The seventh segment steps back to examine the number itself. Chris separates two kinds of 10%. Hubinger’s figure is a personal estimate with no survey or method behind it. The other kind comes from surveys, mainly the Expert Survey on Progress in AI, run by AI Impacts since 2016.

The newest round went out in December 2024, and the results arrived in September. Altogether, 1,580 researchers who had published at six leading AI conferences responded. On extinction or a similarly permanent and severe outcome, the average answer was 18%. Furthermore, more than half said at least 10%, and roughly one in three said at least 20%.

However, the question wording shifts the answers a lot. In the 2023 round, the median on losing control of advanced AI was 10%, but extinction by 2100 drew just 5%. Sample sizes are also smaller than headlines suggest. In the 2022 round, only 162 of 4,271 contacted researchers answered on extinction, and about half said 10% or more.

Some scientists think the press has mishandled all this. Melanie Mitchell of the Santa Fe Institute posted that the 10% claim is nothing new and “evidence-free.” Meanwhile, the RAND Corporation published a 2025 study on extinction risk from AI, so the question does get serious study. Ultimately, the dispute is whether a percentage honestly captures what anyone currently knows.

Six Claims and the Experts Who Push Back

Chris closes with The Guardian, where Robert Booth, Dan Milmo, and Aisha Down tested six recent claims. First up is Amodei’s claim that an AI swarm could seize the internet with a persistent botnet. A botnet is a network of hijacked machines that attacks other systems.

NYU emeritus professor Gary Marcus called the scenario “nonsensical.” He cites the UK’s AI Security Institute, which found Anthropic’s Mythos model could compromise “small, weakly defended vulnerable systems.” That is far from the internet’s backbone. Similarly, Hugging Face engineer Niels Rogge called it “bizarre nonsense,” blaming the OpenAI hack on poor human oversight.

Strikingly, Anthropic co-founder Jack Clark puts dangerous AI about 20 years out, while Amodei says six to twelve months. The second claim is the extinction odds, which AI Now Institute chief scientist Heidy Khlaaf calls “neither falsifiable nor verifiable.” Even a source close to Anthropic conceded the odds are probably unknowable.

Third is the charge that the alarm is a business tactic. Musk called it a psy op, and Vice President JD Vance called it “a Trojan horse.” AI pioneer Yoshua Bengio counters that pacing would cost the companies dearly, an odd thing to fake.

Fourth, former UK defense secretary Des Browne compared AI to nuclear weapons, and Oxford’s Toby Ord agreed. Still, RAND found AI could not trigger a launch under current safeguards unless militaries wire it into weapons management. Chris’s verdict: just don’t put AI into weapons management.

Fifth is the political fight. Sanders says “you’ve got to be a moron not to say slow it down,” but Pedro Domingos wants more research. Sixth is China, which Treasury Secretary Scott Bessent framed as an existential competition. Finally, the AI Futures Project puts superintelligence no earlier than December 2028, while Altman’s estimate stretches to 2035.

Chris’s Take: Healthy Fear and a Kill Switch

After thanking GoDaddy, Chris says he’s fifty-fifty on the debate. Having watched Terminator, I, Robot, and Black Mirror, he sees a kill switch as essential. Going without one, he says, would be ridiculous. Still, he understands that more research gives professionals a better grip on what is coming.

He also argues that fear is healthy. Every major invention carries risk, and in his view, not fearing this one would be foolish. Ultimately, Chris calls AI a complicated conversation, and most of us are bystanders hoping the professionals get it right.

Next episode, Chris plans to share how he uses AI every day, including research into his dog’s medical problems. Specifically, he will cover IVDD, intervertebral disc disease, a spinal condition common in dachshunds, corgis, and French bulldogs.

The post They’re Building Skynet!! #1879 appeared first on Geek News Central.