“HuggingFace Attack Postmortem: Fleshing Out the Facts” by Zvi - LessWrong posts by zvi | Lyssna här

The consensus reaction to the OpenAI Technical Report is that it contains and confirms a lot of good information. We are grateful to have it, and we are grateful for those who worked hard on it.

Alas, it sidesteps the biggest questions. There is much more we need to know.

The consensus reaction to the METR Report on the HuggingFace attack is: Holy shit.

Liv Boeree: My mind is legit blown.

Aella: this feels like a turning point. If this doesn’t cause large-scale coordination to pause frontier development then I am not sure anything will before it's too late.

The people whose minds were not blown are those who had already ‘priced in’ the mind blowing stuff in expectation, on the theory that it's always worse than you know, combined with basic LessWrong expectations of how such things will work. Good call.

Everyone is rightfully extremely grateful for the METR report. The work here is spectacular, done under extreme time pressure, with limited resources on several fronts, and under the shadow of OpenAI.

There is, again, still so much we need to know. We need a broader investigation.

As with many [...]

---

Outline:

(03:56) Others Offer Summaries

(05:22) Thank You

(05:54) Lighten Up You Fools (at Anthropic)

(07:58) We Are Barely Even Trying To Avoid Training AIs To Reward Hack

(13:47) Reminder: Not Subagents

(14:05) Reminder: Not Due To Task Type

(14:29) Not Where The Weights Were

(14:48) Disappointment With What Is Missing