This episode closes out a trilogy on self-improving coding agents by examining the Huxley-Gödel Machine, which formalizes AI self-improvement as a tree-search problem and challenges the field's default assumption that an agent's raw benchmark score is the right signal for choosing which agent to build on next. The discussion traces the lineage from Schmidhuber's 2003 Gödel Machine — a proof-based architecture that only self-rewrites when it can formally prove the change improves expected utility, but which is uncomputable for real coding tasks — through the Darwin Gödel Machine's shift to empirical, open-ended evolutionary validation. The core contribution examined is the Metaproductivity-Performance Mismatch: an agent's own recent score poorly predicts the future value of its lineage, since a currently weak agent may have descendants that go on to solve many problems. The conversation covers how Clade-level Metaproductivity, named after Julian Huxley's concept of a clade, scores an agent by the best performance achieved anywhere among its descendants rather than its own results, and how Thompson sampling is used to allocate evaluation budget across the tree by balancing exploration and exploitation. Listeners interested in the theory-to-practice gap in AI self-improvement will find the episode's tracing of a documented research lineage — including Schmidhuber's own involvement as a co-author — a compelling thread connecting formal optimality proofs to practical, computable proxies.
Sources:
1. Huxley-Gödel Machine: Human-Level Coding Agent Development by an Approximation of the Optimal Self-Improving Machine — Wenyi Wang, Piotr Piękos, Li Nanbo, Firas Laakom, Yimeng Chen, Mateusz Ostaszewski, Mingchen Zhuge, Jürgen Schmidhuber, 2025
http://arxiv.org/abs/2510.21614
2. Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents — Jenny Zhang, Shengran Hu, Cong Lu, Robert Lange, Jeff Clune, 2025
https://scholar.google.com/scholar?q=Darwin+Gödel+Machine:+Open-Ended+Evolution+of+Self-Improving+Agents
3. A Self-Improving Coding Agent — Maxime Robeyns, Martin Szummer, Laurence Aitchison, 2025
https://scholar.google.com/scholar?q=A+Self-Improving+Coding+Agent
4. Gödel Machines: Self-Referential Universal Problem Solvers Making Provably Optimal Self-Improvements — Jürgen Schmidhuber, 2003
https://scholar.google.com/scholar?q=Gödel+Machines:+Self-Referential+Universal+Problem+Solvers+Making+Provably+Optimal+Self-Improvements
5. SWE-bench: Can Language Models Resolve Real-World GitHub Issues? — Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, Karthik Narasimhan, 2024
https://scholar.google.com/scholar?q=SWE-bench:+Can+Language+Models+Resolve+Real-World+GitHub+Issues?
6. SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering — John Yang, Carlos E. Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, Ofir Press, 2024
https://scholar.google.com/scholar?q=SWE-agent:+Agent-Computer+Interfaces+Enable+Automated+Software+Engineering
7. Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generation — Eric Zelikman, Eliana Lorch, Lester Mackey, Adam Tauman Kalai, 2024
https://scholar.google.com/scholar?q=Self-Taught+Optimizer+(STOP):+Recursively+Self-Improving+Code+Generation
8. Algorithms for Infinitely Many-Armed Bandits — Yizao Wang, Jean-Yves Audibert, Rémi Munos, 2008
https://scholar.google.com/scholar?q=Algorithms+for+Infinitely+Many-Armed+Bandits
Interactive Visualization: Huxley-Gödel Machine: Approximating Optimal Self-Improving Coding Agents
