Redwood Research Blog
Avsnitt

“Untrusted advice for AI control: Short, strong advice significantly uplifts weak LLMs” by Caleb Biddulph, Adam Kaufman

Redwood Research Blog
Dela

Subtitle: When a misaligned AI can only output tiny amounts of information, it may find sabotage very difficult.

TL;DR: We introduce the untrusted advice protocol, in which a trusted executor LLM takes every action and an untrusted advisor LLM can only send it short hints. Even with as few as 4 characters per step, this advice recovers a substantial fraction of the capability gap between the two models. Because the untrusted LLM's influence flows through such a narrow, monitorable channel, we argue that this achieves near-maximal safety in our BashArena setting. We also discuss the general concept of information bottlenecks and their benefits for interpretability, security, and cost.

In SWE-bench Verified, a strong, untrusted LLM advising a weak, trusted LLM every step can significantly improve the latter's performance, even when we limit the length of the advice. See the more detailed version of this figure later in this post.

In high-stakes AI control, we want to safely use a highly capable but untrusted model (U) that might secretly attempt a misaligned, catastrophic action. To do this, we create protocols that call U alongside a less capable, trusted model (T). Typically, T takes an auxiliary role in these protocols [...]

---

Outline:

(05:01) Experiments

(05:36) Main experiment: how does limiting advice length affect performance?

(09:55) Reducing U's bit usage

(11:01) Counting bits using LLM surprisal

(13:48) Making U select from finite options

(14:28) Why don't we red-team this protocol?

(17:08) Is studying maximally safe protocols worth the safety tax?

(19:16) Types of restrictions on U's advice

(21:20) Information bottlenecks provide other advantages

(21:52) Interpretability

(24:01) Security

(24:26) Cost

(25:14) Conclusion

(26:28) Appendix: more ways to implement information bottlenecks

(26:34) Amortizing U's influence with pre-deployment work

(28:25) Interpolating between T and U

(29:04) Bottlenecking updates to T's weights

(31:16) Appendix: colluding instances of U could defeat untrusted advice

(33:21) Appendix: how to measure surprisal

(38:11) Appendix: selecting advice from a menu

(40:44) Appendix: best-of-n protocol

(42:37) Appendix: advising less frequently

The original text contained 26 footnotes which were omitted from this narration.

---

First published:
July 27th, 2026

Source:
https://blog.redwoodresearch.org/p/untrusted-advice-for-ai-control-short

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Diagram showing untrusted advice protocol with information bottleneck between two robots.In SWE-bench Verified, a strong, untrusted LLM advising a weak, trusted LLM every step can significantly improve the latter’s performance, even when we limit the length of the advice. See the more detailed version of this figure later in this post.The bold curve pools the two trusted executor LLMs (Gemini 3.1 Flash Lite and gpt-oss-120b), and the two faint curves behind it are from the individual LLMs. The shaded band is a 95% confidence interval from a paired task-level bootstrap.These are the same pooled usefulness curves as above, but the x-axis now counts bits of advice instead of characters. The gray curve uses the naive Unicode bound of 8 bits per character, and the green curve uses Llama-3.3-70B-Instruct-Turbo’s surprisal conditioned on T's context. Error bars are 95% confidence intervals from a paired task-level bootstrap. The gray curves use the same data as the curves in the previous figure, but they look slightly different: for certain datapoints, <20% of tasks were dropped due to the surprisal LLM’s context limit.Bits per character of advice as a function of the per-message character cap, for both trusted executor LLMs and both trusted surprisal LLMs, comparing naive Unicode bit-counting to the three surprisal-based methods. Error bars are 95% confidence intervals from a paired task-level bootstrap.Usefulness vs. bits of advice per task, adding the menu-based protocol (stars) to the freeform untrusted-advice curves (circles). We use our best surprisal-based method (Llama’s surprisal with T’s context) to count bit usage in our untrusted advice experiments.Per-task BashArena score distributions for T alone (Gemini 3.1 Flash Lite), best-of-16 where U picks between 16 actions independently sampled from T, and U alone (Claude Sonnet 4.6). The diamonds and bold numbers are mean scores, and the error bars are 95% confidence intervals of the mean from a bootstrap over tasks.Usefulness vs. characters of advice per task in BashArena, comparing the default “advice every step” protocol to the two approaches described above for limiting the cadence of U’s advice. The data comes from early experiments that used different values for the max characters per message.

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Rss Apple Podcaster
Podden och tillhörande omslagsbild på den här sidan tillhör Redwood Research. Innehållet i podden är skapat av Redwood Research och inte av, eller tillsammans med, Poddtoppen.