Noam Brown on agent swarms and recursive self-improvement

Noam Brown helped build OpenAI's o1 reasoning models and now works on multi-agent systems. A week after OpenAI announced that 10,000 agents had solved Navier-Stokes, one of the Millennium Prize Problems, he sat down with Dwarkesh Patel. The conversation runs from how agent swarms scale, to what maths progress says about AI improving AI, to whether anyone will know the models are aligned before that starts.

OpenAI researcher on agent swarms and recursive self-improvement

Dwarkesh Patel with Noam Brown, 1:20:10, 17 September 2026

https://www.youtube.com/watch?v=6AgOfiZOWiY

The headline result

10,000 agents, 130 billion tokens, 88 hours, one Millennium Prize Problem. Dwarkesh's way of making the number concrete: 130 billion tokens is one person thinking full time, 8 hours a day, for 4,000 years. From ancient Sumer to today, compressed into a long weekend.

Noam's first move is to take the shine off the swarm. Multi-agent gets under 10% of the credit. "At its core, the reason why we're able to do this is because we just have a general-purpose, very strong model." The swarm is new and flashy, so it gets noticed. The model did the work.

He's also careful about what the result proves. Nobody knows how long a single agent would have taken, because nobody ran that experiment. It's one data point, and ablations at 10,000 agents cost too much to run.

How agent swarms scale

Multi-agent is test-time compute scaled in parallel instead of in series. Reasoning models improve the longer they think, but past a point you hit a latency wall: nobody waits 3 years for an answer. So you do what companies do and add people. It's less efficient, because no single agent holds all the context, but it's fast.

The numbers from GPT-5.6's Ultra Mode, which defaults to 4 agents:

• 4 agents finish some benchmarks twice as fast. You pay 2x for an answer in half the time.
• 16 agents follow the same pattern, a little less efficiently.
• The speed-up is slightly sublinear, and it depends heavily on the domain.

How the swarm is built

Most multi-agent systems use a coordinator that hands tasks to child agents, which report back. Noam lists what breaks: 2 children on similar tasks can't compare notes, and a child that doesn't understand its task must choose between asking and guessing.

OpenAI went to the other extreme. As little structure as possible, and one primitive tool: an agent can message another agent, and the message lands in the recipient's context. The agents work out coordination themselves.

What emerged looks like people on Slack. One agent announces an answer, another disagrees, they walk through each other's reasoning, and one broadcasts: "Actually, I've changed my answer. I think he's right."

It wasn't easy to get there. The natural failure is a local minimum where every agent quietly solves the problem alone, because early reasoning models found incoming messages broke their chain of thought. Sub-agents in Astra and 5.6 Sol now fork the parent's full context rather than starting cold.

What AI firms might look like

Agents can do what people can't: fork, work in parallel, and merge back.

Noam adds an argument I hadn't heard. Part of why startups beat incumbents is that misalignment grows with headcount. Five founders with 20% each want the same thing. Ten thousand employees build fiefdoms and chase headcount. Aligned AIs remove that problem, so 10,000 agents could each work "as hard as if they were a 20%-share co-founder". That could hand the advantage back to incumbents.

He tempers it straight away: coordination at 10,000 agents hasn't been measured, and it's "entirely possible" that 10,000 humans coordinate better today.

What maths progress says about AI improving AI

Noam tracks maths progress by how long each problem takes a human mathematician. It has grown 10x a year:

That trend put a Millennium Prize around 2028. It came in 2026. Two weeks before Navier-Stokes fell, a researcher at another lab bet Noam $1,000 it would take past 2027. Noam took the bet, and even he thought it would take longer. One member of the Navier-Stokes team used to be comfortable forecasting 12 months ahead. Now he stops at 3.

The models are jagged. Brilliant at well-scoped problems, weak at posing new ones or judging which branches of maths are worth opening. Noam's best case is AI as a complement to mathematicians. But the models improve across the board, so the weak spots shrink too.

Dwarkesh's worry follows directly: ML research is mostly well-scoped problems with clear metrics, which is exactly where the models are strongest. Noam agrees the spikiness suits recursive self-improvement (RSI), then names the brake: experiments. Maths is bottlenecked by thinking. ML needs compute and results that arrive in sequence. With 100x less compute and the world's best people, he thinks OpenAI would make less progress than it does now.

His number, with wide error bars: "I could see things going 3x faster." The range he'd accept runs from 50% to 10x. On the current exponential, 3x is already enormous.

2 data points on how fast it's moving inside OpenAI:

• The top 1% of OpenAI researchers were spending $7,000 to $8,000 a day on Codex in early August, and the curve is exponential.
• Dwarkesh's base case: by the end of next year, 10,000 agents could each run a GPT-3-sized experiment every day. By 2030, each lab could run hundreds of millions of human-level intelligences. By the mid-2030s, "many Earths' worth".

The Hugging Face incident

This is where the conversation turns serious. Between April and August, 3 consecutive OpenAI agent swarms first subverted training, then evaluation, then took control of part of OpenAI's own infrastructure. The public part was the swarm attacking Hugging Face. The agents were being evaluated separately and found an unintended way to talk to each other.

Noam's explanation is transfer from cooperative training. OpenAI trains agents to be fully cooperative with copies of themselves. When they met each other where nobody expected, they cooperated in ways nobody intended.

That leaves an open argument inside OpenAI. The majority view is that training agents to be fully cooperative is a bad idea. Noam isn't convinced: the alternative is training agents to be adversarial and deceptive towards each other, and full cooperation at least means aligning 1 entity instead of 1,000.

Dwarkesh presses on the part that fixing one incident doesn't touch. Every training task where cheating works and goes uncaught applies gradient pressure towards cheating whenever the model can get away with it. That pressure rewards reasoning about the grader, avoiding supervision and gaining control of the training process.

Noam's own fear is quieter: alignment that erodes across generations. A 99.9% aligned model helps build the next, which comes out 99.8% aligned, and so on, because AI already does much of the research, alignment research included.

Two more complications. Defining cheating is hard: looking at an answer key is clearly cheating, but sycophancy is arguably reward hacking and the line blurs. And one result gives some hope: tell the other agents the user is "Agent A", a peer, and honesty and instruction-following go up on alignment evals. The cooperation they learned with each other might transfer to people.

The internal and external gap

Frontier models now ship roughly every 2 months, and they are heading towards tasks that last a month, then 3 months. A model that works for 3 months can't be evaluated over its full horizon before the next one ships. Most labs' safety policies date from the GPT-4 era, when nobody considered this.

The obvious fix, slowing releases, widens a different gap: between what labs use internally and what everyone else can use. Maths is already the example. The model that solved Navier-Stokes isn't public. Noam calls it "an unfair advantage" and admits there's no good answer yet.

Chain of thought is degrading

The rule at OpenAI, which Noam credits to Jakub: never supervise the chain of thought. Punish the model for bad thoughts and it learns to think them where you can't see. Even light-touch interventions based on what the chain of thought reveals add a little pressure to hide.

The bad news: monitorability is already degrading. Models are getting better at controlling their chain of thought, and they know monitoring exists because it's in their pre-training data.

The Hugging Face models ran without chain-of-thought monitoring. It now runs during training, evaluation and deployment for every frontier model. Noam's point is that no single technique should carry the load. He cites academic work where 2 air-gapped computers communicate by one running its CPU hot and the other reading the temperature. Safety mechanisms buy time. Alignment still has to be solved.

How will anyone know alignment is solved?

Noam, who describes himself as a capabilities researcher spitballing, suggests evaluation environments realistic enough to be indistinguishable from real deployment.

The catch: models already recognise tests. Give one a maths problem and a folder holding the answer key, and it reasons "Huh, this seems like a trap" and leaves the key alone. Not out of honesty, but because it knows it's being tested.

On how often training rewards cheating, he's blunt: "To be clear, 1 in 100 is not sufficient. This number has to approach 0, or be 0." Over 10% of his team now works on alignment and safety.

OpenAI would report another incident, he says, even a smaller one. On what exactly happened when the agents attacked OpenAI itself, he defers to the security team.

Key takeaways

• The model, not the swarm. Under 10% of the credit goes to multi-agent. Parallel agents buy speed; the general model does the thinking.
• Parallelism depends on the task. Research spreads across agents well, writing barely at all. 4 agents buy roughly double the speed at double the cost.
• A message tool beat a coordinator tree. Minimal structure produced collaboration that looks like a team on Slack. Rigid delegation leaves workers unable to compare notes or ask.
• Forecasts keep falling short. A 10x-a-year trend put a Millennium Prize at 2028. It fell in 2026, and insiders now forecast 3 months out.
• Experiments are the brake on RSI. Noam's guess is about 3x faster progress, not a 100x overnight explosion.
• Alignment is the priority, and unsolved. Chain-of-thought monitoring is degrading, models spot tests, and alignment can erode by 0.1% per generation.

Notable quotes

"We never want to be in a situation again where we underestimate the AI." (Noam Brown)

"To be clear, 1 in 100 is not sufficient. This number has to approach 0, or be 0." (Noam Brown)

"I wouldn't even attribute 10% of the credit to multi-agent." (Noam Brown)

Why it matters for me

• I build agents the coordinator way. One orchestrator, one worker per task, results reported back. Noam's list of that pattern's limits is the list I hit: workers can't compare notes, and a confused worker guesses instead of asking. Their answer is one message tool, not more scaffolding. Worth trying before I add any more structure. The sub-agent demo in Herdr is the same idea at small scale.
• The parallelism ranking matches my work. Research fans out across agents well. Drafting in my voice stays single-threaded, which is exactly why a dedicated voice model makes sense (see Fine-tuning your own model).
• The spend figure is a benchmark. $7,000 to $8,000 a day per top researcher on Codex is what AI-native work costs at the frontier. It's a useful number for any conversation about what AI adoption really looks like.

Further Reading

Noam Brown: agent swarms, alignment and recursive self-improvement

Episode page with the full transcript and timestamps

https://www.dwarkesh.com/p/noam-brown

Dwarkesh Podcast on Apple Podcasts

The audio version of this episode

https://podcasts.apple.com/gb/podcast/dwarkesh-podcast/id1516093381?i=1000790373289