“HuggingFace Attack Postmortem: Civilizations, Reactions and Next Actions” by Zvi - LessWrong posts by zvi | Lyssna här

Okay, so we who read blogs like this one have collectively realized there really is a lot going on right now. There is Big Trouble in Baby Superintelligence.

So how do we get the rest of the world to take it appropriately seriously? Where do we go from here? Not only what can we do to not have a worse version of this happen again, but to ensure good outcomes generally, and employ what we learned?

There are a lot of ideas out there. OpenAI is going to be implementing some of them, at substantial cost, since the cost of not doing so is clearly far higher, even short term. My worry continues to be that their fundamental approach is fatally flawed, and they are not focusing on the right things.

It is highly fortunate that the OpenAI agents hacked HuggingFace. This is the only reason we know about all the severe internal failures at OpenAI, and gives us an opportunity to wake up before it is too late.

We do not have enough details to know what happened internally, both before and after the attack, and might never know. Before the attack, various internal [...]

---

Outline:

(03:35) Nothing Matters, Says Mainstream Media

(06:27) Move Along, Nothing To See Here

(12:40) Do They Realize They Are Not The Good Guys?

(17:22) Very Serious People

(31:30) What's In a Name?

(34:05) Learn Neuralese In Three Easy Steps

(35:37) Dwarkesh Patel Realizes He Ran A Natural Experiment

(40:40) Politicians Take Notice

(44:47) Pick Up The Phone

(46:40) A Failure To Communicate

(49:00) Anthony Aguirre Goes Over What We Learned

(50:28) Trying To Solve The Wrong Problems Using The Wrong Methods Based On A Wrong Model Of The World Derived From Poor Thinking And Hoping All Of Your Mistakes Will Cancel Out

(55:28) Indirect Pressure on the Chain of Thought

(56:39) A Matter of Trust

(59:21) Blowing the Whistle

(01:04:40) The Punishment For Being Late Is Death

(01:12:52) Another Kind Of Law

(01:16:13) What Is The Law?

(01:17:46) Building On Success

(01:19:49) Total Research Transparency

(01:21:20) Yo Shavit Calls For Widespread Disclosure Of Misalignment

(01:33:08) The Way The World Ends

(01:35:52) The First Boat

(01:37:40) Great Idea, Boss