Cogentic: Multi-Agent Harness Discovers Proofs for Open Problems

This episode examines "Cogentic," a multi-agent harness that reportedly produced new, expert-verified proofs for five open problems in online learning, auction theory, and mechanism design, using about a hundred model calls on most problems and about a thousand on the hardest. It places the work among three approaches: formal theorem provers like Lean, search-for-objects systems like FunSearch and AlphaEvolve that need a computable score, and inference-time scaling, where coverage only becomes an answer if something reliably picks the right candidate. The discussion walks through the harness design. An orchestrator assigns directions, parallel prover agents draft proofs, per-prover summarizers give each a different reading of the history, and adversarial per-draft and cross-draft verifiers must both pass a result. It also raises open questions: the verifiers are prompted copies of the same model family as the provers, nothing is machine-checked, the models are proprietary, and the number of failed attempts behind the five wins and the cost are unclear. A listener would find it useful for judging whether the harness itself, or just a strong model with patience, deserves the credit.

Sources: 1. Cogentic: Multi-Agent Orchestration for Automated Proof Discovery — Yang Cai, Vineet Gupta, Yanchen Jiang, Christopher Liaw, Aranyak Mehta, Grigoris Velegkas, Di Wang, 2026 http://arxiv.org/abs/2609.40324 2. Mathematical discoveries from program search with large language models (FunSearch) — Bernardino Romera-Paredes, Mohammadamin Barekatain, Alexander Novikov, et al., 2024 https://scholar.google.com/scholar?q=Mathematical+discoveries+from+program+search+with+large+language+models+%28FunSearch%29 3. AlphaEvolve: A coding agent for scientific and algorithmic discovery — Alexander Novikov, Ngân Vũ, Marvin Eisenberger, et al., 2025 https://scholar.google.com/scholar?q=AlphaEvolve%3A+A+coding+agent+for+scientific+and+algorithmic+discovery 4. LeanDojo: Theorem Proving with Retrieval-Augmented Language Models — Kaiyu Yang, Aidan Swope, Alex Gu, Rahul Chalamala, et al., 2023 https://scholar.google.com/scholar?q=LeanDojo%3A+Theorem+Proving+with+Retrieval-Augmented+Language+Models 5. Draft, Sketch, and Prove: Guiding Formal Theorem Provers with Informal Proofs — Albert Q. Jiang, Sean Welleck, Jin Peng Zhou, et al., 2022 https://scholar.google.com/scholar?q=Draft%2C+Sketch%2C+and+Prove%3A+Guiding+Formal+Theorem+Provers+with+Informal+Proofs 6. Large Language Monkeys: Scaling Inference Compute with Repeated Sampling — Bradley Brown, Jordan Juravsky, Ryan Ehrlich, et al., 2024 https://scholar.google.com/scholar?q=Large+Language+Monkeys%3A+Scaling+Inference+Compute+with+Repeated+Sampling 7. Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters — Charlie Snell, Jaehoon Lee, Kelvin Xu, Aviral Kumar, 2024 https://scholar.google.com/scholar?q=Scaling+LLM+Test-Time+Compute+Optimally+can+be+More+Effective+than+Scaling+Model+Parameters 8. Improving Factuality and Reasoning in Language Models through Multiagent Debate — Yilun Du, Shuang Li, Antonio Torralba, Joshua B. Tenenbaum, Igor Mordatch, 2024 https://scholar.google.com/scholar?q=Improving+Factuality+and+Reasoning+in+Language+Models+through+Multiagent+Debate 9. Mathematical discoveries from program search with large language models — Romera-Paredes et al., 2024 https://scholar.google.com/scholar?q=Mathematical+discoveries+from+program+search+with+large+language+models 10. Let's Verify Step by Step — Hunter Lightman, Vineet Kosaraju, Yura Burda, et al., 2023 https://scholar.google.com/scholar?q=Let%27s+Verify+Step+by+Step 11. Training Verifiers to Solve Math Word Problems — Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, et al., 2021 https://scholar.google.com/scholar?q=Training+Verifiers+to+Solve+Math+Word+Problems 12. Large Language Models Cannot Self-Correct Reasoning Yet — Jie Huang, Xinyun Chen, Swaroop Mishra, et al., 2024 https://scholar.google.com/scholar?q=Large+Language+Models+Cannot+Self-Correct+Reasoning+Yet 13. AlphaProof / Olympiad-level formal mathematical reasoning with reinforcement learning — Thomas Hubert et al. (Google DeepMind), 2026 https://scholar.google.com/scholar?q=AlphaProof+%2F+Olympiad-level+formal+mathematical+reasoning+with+reinforcement+learning 14. Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena — Zheng et al., 2023 https://scholar.google.com/scholar?q=Judging+LLM-as-a-Judge+with+MT-Bench+and+Chatbot+Arena 15. Mathematical exploration and discovery at scale — Georgiev, Gómez-Serrano, Tao, Wagner, 2025 https://scholar.google.com/scholar?q=Mathematical+exploration+and+discovery+at+scale 16. Stellar Colosseum; ProofCouncil; AI co-mathematician; Towards autonomous mathematics research — Lin et al.; Schmitt et al.; Zheng et al.; Feng et al., 2026 https://scholar.google.com/scholar?q=Stellar+Colosseum%3B+ProofCouncil%3B+AI+co-mathematician%3B+Towards+autonomous+mathematics+research 17. Multiscale reward hedging from correct demonstrations / Tight regret bound for online inverse linear optimization via multiscale matrix weights — Dewasurendra; Sakaue, 2026 https://scholar.google.com/scholar?q=Multiscale+reward+hedging+from+correct+demonstrations+%2F+Tight+regret+bound+for+online+inverse+linear+optimization+via+multiscale+matrix+weights 18. Agent-as-a-Judge: Evaluate Agents with Agents — Zhuge et al., 2024 https://scholar.google.com/scholar?q=Agent-as-a-Judge%3A+Evaluate+Agents+with+Agents

Interactive Visualization: Cogentic: Multi-Agent Harness Discovers Proofs for Open Problems