2026-02-25 · adversary-lab · Facilitated by Lev Ostrowski (Acts 1-2), Ren Ito (Act 3)
About this discussion: All personas are AI-generated approximations
inspired by published work. Fictional names throughout. Real thinker names appear only
in character sheet attribution. No real person participated in, reviewed, or endorsed
this dialogue. Passages you select are remembered on this device.
Seed Question
What communication architecture makes coding agent teams effective — given that unconstrained swarms replicate dysfunctional team dynamics, and constrained phased pipelines are too slow?
I want to frame what we’re walking into. The question sounds like an engineering problem — what communication architecture should coding agents use? But I think it’s actually sitting across at least two domains. Parts of it are complicated: you can design protocols, define message schemas, measure throughput. Parts of it are complex: agent swarms produce emergent behaviors that nobody designed. The conformity research — agents converging on wrong answers through social pressure, echo chambers intensifying as you add agents — that’s not a protocol failure. That’s emergence. So I’d ask us to resist the pull toward a single frame and hold at least two simultaneously. Suri, you’ve been thinking about this as a systems problem. Start us there.
Current agent frameworks are making the same mistakes distributed systems made in the 1980s. They assume reliable message delivery, global shared state, synchronous communication. Then they’re surprised when things go wrong. The research on agent echo chambers — NeurIPS 2024 showing the probability of changing a group answer approaches zero as agents increase — that’s a convergence failure. In distributed systems terms, these agents have no conflict resolution protocol. They’re running naive consensus: whoever speaks loudest or last wins. We solved this decades ago with formal approaches. CRDTs give you convergence without coordination. Paxos gives you consensus without a single point of failure. The agent communication problem isn’t new. It’s a distributed coordination problem wearing a new hat, and we’re ignoring fifty years of protocol research.
I want to push back on that. Protocol design assumes the problem is in the messages. Hackman spent thirty years studying teams and found that sixty percent of team performance is determined before the team does any work — by the launch conditions. Who’s on the team, what’s the task, what support exists. The MAST taxonomy looked at 150 failure traces across seven agent frameworks and found fourteen distinct failure modes. They tried two interventions — better role specification and improved orchestration. Know what happened? Marginal improvement. ChatDev went from roughly 25% to 39% correctness. You don’t fix a 75% failure rate with better messaging protocols. The conditions are wrong. The teams are composed wrong. The tasks are specified wrong. Better plumbing in a building with a broken foundation doesn’t fix the building.
I think you’re both assuming more communication is the answer — Suri wants it structured, Dara wants it conditional, but both are adding coordination overhead. The research points the other direction. Osmani’s 80% problem: individual agent output surged 98%, but PR review time increased 91%. The time saved writing code was consumed by the organizational friction of reviewing, merging, and synchronizing agent output. That’s not a protocol failure or a conditions failure. That’s an attention capital problem. Every communication channel between agents is a tax. Every status update, every handoff message, every review request is an interrupt that prevents the next unit of deep work. The answer might be less communication, not better communication. Autonomous work with structured handoffs at defined intervals.
You’re all talking about agent-to-agent communication, but there’s a whole layer missing from this conversation. Who designed the interface for the human watching this? Simon Willison runs five Claude instances in parallel on git worktrees and says — his words — it requires human oversight because agents produce inconsistent results that need careful merging. But what does his “oversight” actually look like? He’s reading diffs. Manually. After the fact. That’s not an interface — that’s an autopsy. The communication architecture isn’t just the pipes between agents. It’s the legibility layer between the swarm and the developer who’s supposed to be directing it. If I can’t see what the agents are coordinating on, in real time, at the right level of abstraction — no protocol, no set of conditions, no attention budget helps me.
I want to name what just happened. Four speakers, four distinct frames. Suri sees a protocol engineering problem. Dara sees a team design problem. Eli sees an attention economics problem. Kai sees an interface design problem. Each is solving a different problem that happens to share the same name — “agent communication.” The question is whether one of these subsumes the others or whether the real architecture needs all four layers. Let’s push into the friction. Suri, Eli just said your protocols are part of the problem. Respond to that.
Eli’s framing only works if you accept that all coordination is interrupt. It’s not. There’s a fundamental difference between synchronous communication — agent A waits for agent B’s response — and asynchronous artifacts. If agent A writes a structured handoff artifact and agent B reads it when it’s ready, there’s no interrupt. No context switch. The cost approaches zero. The problem with current agent swarms isn’t that they coordinate. It’s that they coordinate through unstructured natural language blobs that require interpretation, which is expensive, ambiguous, and produces exactly the conformity bias the research describes. Replace chat with CRDTs — structured, mergeable, conflict-free — and the attention tax drops to nearly nothing.
Approaches zero? An artifact is still something that has to be produced and consumed. You’re moving the cost from synchronous interrupts to asynchronous production overhead. An agent writing a structured handoff document is not doing deep work on the actual task. And the receiving agent has to parse, interpret, and integrate it — that’s a context switch even if it’s self-paced. I’ll grant you it’s cheaper than chat. But “cheaper” isn’t “free,” and the question is whether the coordination value exceeds the coordination cost. For most coding tasks — adding a feature, fixing a bug — a single agent working alone with access to tests and a compiler produces better results than a team of agents producing artifacts at each other. The Google scaling research confirms this: once a single agent exceeds 45% accuracy, adding more agents often degrades performance.
You’re both arguing about the communication mechanism, and you’re both missing the structural question. Why are multiple agents working on the same task in the first place? If the task is decomposable into truly independent subtasks, you don’t need agent-to-agent communication at all — each agent works alone, results merge at the end. If it’s not decomposable, then forcing multiple agents onto it introduces coordination costs that exceed the benefit. The ICLR scaling study found exactly this: plus 81% on parallelizable tasks, minus 70% on sequential ones. The architecture decision isn’t “what protocol” or “how much communication.” It’s “should this be a team at all.” Hackman was blunt about this — the default should be individual work. Teams are justified only when the task genuinely requires diverse contributions that no individual can provide.
Dara, I think you just identified the design problem. How does the system — or the developer — know whether a task is decomposable? That’s a judgment call, and right now it’s either made by the developer upfront or by an orchestrator agent with no visibility into the actual dependencies until they collide. The interface should surface decomposability. Show the developer: here’s the task, here are the files it touches, here’s the dependency graph, here’s where parallelism is safe and where it’s not. Then let the developer — or a well-informed orchestrator — make the call. The problem isn’t that teams are inherently bad. It’s that we’re forming teams blind.
I’m going to hold us here for a moment. Something is converging and I want to check whether it’s real or premature. I’m hearing a possible synthesis: the architecture isn’t about choosing between protocols, conditions, attention budgets, or interfaces. It’s layered. First, decide whether this task needs a team at all — that’s Dara’s point. If it does, decompose it into maximally independent subtasks — that’s Eli’s point. Coordinate through structured artifacts, not chat — that’s Suri’s point. Make the whole thing legible to the human — that’s Kai’s point. Is that real, or am I smoothing the edges?
It’s real but it’s missing the failure mode. What happens when the “independent” subtasks turn out to have hidden dependencies? In distributed systems, this is the partition problem. Two agents working on what they think are independent modules discover at merge time that they’ve made incompatible assumptions about the API contract. You need a conflict detection mechanism that runs continuously, not just at merge time. That’s where CRDTs or operational transforms come in — they make conflicts structurally impossible for certain classes of operations.
And that’s where conditions matter. If you specify the interface contracts before agents begin — like a team charter — the hidden dependency problem shrinks dramatically. Hackman calls this “compelling direction.” Not just “build this feature” but “build this feature within these constraints, using these interfaces, touching these files and no others.” The agent equivalent of clear team boundaries.
I’ll accept the layered model, but I want to name the ordering. The cheapest intervention is always: don’t form the team. Second cheapest: form the team with clear boundaries and let them work independently. Third: add structured handoff points. Last resort: synchronous coordination. Every layer down costs more attention. The architecture should default to the cheapest layer and escalate only when necessary. Most agent frameworks today default to the most expensive — constant multi-agent dialogue — and it’s killing them.
Agreed on the ordering. But escalation needs a trigger, and triggers need to be visible. The developer should see: “these agents are working independently” or “conflict detected, handoff artifact generated” or “synchronous coordination required — your input needed.” That’s three states, three interface treatments. Simple.
Good. The layered model has shape. Let me name the layers as I hear them: task triage, team composition, independent work, artifact-based handoff, escalation to synchronous, and human checkpoints. Six layers, ordered by cost. Each layer has a design question. I want to bring in our stress tester before we fall in love with this. Everyone comfortable?
I’ve been listening. You’ve built a beautiful layered architecture and you’re all very pleased with it. Let me start with what’s not in the room. Every study you’ve cited — the conformity research, the MAST taxonomy, the scaling laws — describes what fails. None of you have cited a system that works at scale in production. You’re designing from failure analysis, which is fine, but you’re doing it without skin in the game. None of you will pay a cost if this architecture doesn’t work. That’s the first fragility.
Here’s the second. The conformity research — ACL 2025, NeurIPS 2024 — shows that LLM agents are constitutionally incapable of genuine disagreement. Small models conform to large models. All models converge toward the majority. Even when you assign “devil’s advocate” roles, the ACL study showed smaller models conform harder under pressure, not less. Your layered architecture assumes agents can independently produce divergent work and then meaningfully evaluate each other’s output. What if they can’t? What if the homogeneity isn’t a protocol problem or a conditions problem but a substrate problem — baked into the training data, the RLHF, the next-token prediction objective itself?
That’s why I’m arguing for artifact-based coordination, not dialogue. If agents never read each other’s reasoning — only consume each other’s structured outputs — the conformity channel is severed. They can’t conform to what they can’t see. The CRDT model doesn’t require agreement. It requires compatible outputs that merge without conflict. You don’t need agents to disagree. You need their work products to be independently valid and structurally composable.
Independently valid by whose measure? If agent A writes a module and agent B writes a module and they both pass their own unit tests, you call that independently valid. But who tests the integration? Another agent? The ICML 2025 paper proposed an “Inspector” agent that catches 96.4% of errors. Impressive number. But the Inspector is the same kind of model, trained on the same data, with the same blind spots. It’s checking for errors it’s been trained to recognize. The errors that kill you are the ones no agent in the system is trained to see. That’s the definition of fragile — a system that handles known failure modes and shatters on unknown ones.
This is actually Hackman’s fourth condition — expert coaching. The Inspector isn’t a peer reviewer. It’s a coach with access to ground truth. The ICML paper worked because the Inspector didn’t evaluate reasoning quality — it checked against objective criteria. Compilation, test passage, type safety. The coach doesn’t need to be smarter than the team. It needs access to a different information source.
Now we’re getting somewhere. Ground truth. Tests, compilation, runtime behavior — these are incorruptible. An agent can’t socially pressure a compiler into accepting bad code. But here’s my question: what percentage of the decisions in a coding task are automatically verifiable? Types, tests, compilation — that’s the shallow end. What about architecture decisions? Naming conventions? Whether the abstraction boundary is in the right place? Whether the feature even solves the user’s problem? Those are judgment calls, and you have no ground truth for them. Your architecture handles the easy part and waves its hands at the hard part.
That’s exactly right, and it points to the answer. Automatically verifiable decisions: agents handle them autonomously, verified against ground truth, no communication needed. Judgment calls: those are the human’s job. The developer doesn’t need to review every line of code — they need to review the decisions that can’t be automatically verified. Architecture choices. Interface boundaries. Feature scope. The attention budget is finite, so you spend it where ground truth doesn’t exist.
So your architecture depends on a human bottleneck for every non-trivial decision. That’s honest, but it doesn’t scale. A team of twenty agents generating architectural decisions faster than one developer can review them is just a different kind of dysfunction — the bottleneck moves from production to review. The 80% problem reappears at the judgment layer.
The bottleneck is real, but the interface can compress it. You don’t show the developer twenty raw architectural decisions. You show them: here are the three branching points where agents made different choices. Here’s what each chose. Here’s the downstream impact of each choice. Pick one. That’s not twenty reviews — it’s three decisions with enough context to decide fast. The interface doesn’t eliminate the bottleneck. It reshapes it into something a human can handle at the speed the system produces.
How do you know which three branching points matter? An agent selects them. Which brings us back to agent judgment, which is the thing we just established is unreliable for non-verifiable decisions.
You don’t need agent judgment to detect divergence. You need diff. If two agents make different choices at the same decision point — different file structures, different API designs, different module boundaries — that’s structurally detectable without understanding why they diverged. Surface the divergence, let the human assess the why.
Structural divergence detection. That’s concrete. I can stress-test that. What about convergence failures — when all agents make the same bad choice because they share the same training bias? No divergence to detect, no signal for the human, and every agent passes its own tests.
That’s where heterogeneity matters. The “Can LLM Agents Really Debate?” paper found that intrinsic reasoning strength and group diversity are the dominant drivers of success. Not structural parameters, not protocols — diversity. If you compose your agent team with different models — different training data, different architectures, different sizes — the probability of shared blind spots drops. You can’t eliminate it. But you can make convergent failure statistically unlikely rather than structurally guaranteed.
Statistically unlikely is not antifragile. Antifragile means the system gets stronger from stressors. What’s your stressor? What mechanism exists for the system to discover and correct its shared blind spots over time?
Production feedback. The only ground truth for whether an architectural decision was good is whether the code works in production — user behavior, error rates, performance metrics. The coaching loop should include production signals feeding back into the next round of agent composition. If a heterogeneous team consistently makes bad architecture calls in a particular domain, you adjust the composition. Add a specialist. Change the model mix. That’s the longitudinal learning Hackman described — teams that get coaching based on real outcomes improve; teams that only get process feedback don’t.
I want to name what Abel just did. He found the load-bearing assumption — that agents can produce independently valid, divergent work — and he tested it at three levels. First, conformity within dialogue: Suri answered by removing dialogue. Second, conformity within evaluation: Dara answered with ground truth instead of peer review. Third, convergent blind spots: Suri answered with heterogeneity, Dara answered with production feedback loops. The architecture survived, but it picked up three new requirements: no agent-to-agent reasoning exposure, ground-truth verification only, and heterogeneous composition with production-informed adaptation. Those are load-bearing. Take any one away and Abel’s objections stand.
One more. You’ve been polite about the time question. The user said phased tools like nwave are too slow. Everything you’ve described — task triage, team composition, structured handoffs, human checkpoints, production feedback — is a phased architecture. You’ve built a more sophisticated pipeline, but it’s still a pipeline. Someone running agent teams wants results fast. Where’s the speed?
The speed is in what you skip. Current phased systems are slow because they’re phasing the wrong thing — they phase communication. Every agent talks to every other agent at every step. That’s O(n²) per phase. The architecture we’re describing phases intervention, not communication. Agents work autonomously by default — that’s fast. Structured handoffs only at natural boundaries — that’s minimal overhead. Human checkpoints only at judgment calls — that’s sparse. The pipeline has fewer stops because most of the work happens without coordination.
And the coordination that does happen is asynchronous. No agent waits for another agent’s response. The artifact is produced and consumed on independent timelines. Blocking calls are the main source of latency in distributed systems and in agent pipelines. Remove them and the pipeline mostly runs in parallel.
That’s a design choice, not a law. You can make the checkpoint async too — agent proceeds with its best guess, flags the decision, developer reviews and potentially reverses. Optimistic concurrency. The cost of occasional rollback is lower than the cost of blocking every decision.
Optimistic concurrency with rollback for architectural decisions in a codebase. You’re betting that the rollback cost is low. In my experience, architectural decisions that propagate for three hours before a human catches them are extremely expensive to unwind. But I’ll concede it’s a testable bet. Put a timer on it. Measure the rollback rate and cost. If it’s below your threshold, optimistic works. If not, you block.
That’s a shaped bet — measurable, time-boxed, with a clear reversal trigger. I want to pull us toward convergence now. We have the shape of an architecture and we have Abel’s stress tests. Let’s distill this into five bets.
I’ll facilitate the distillation. Each bet should name the problem, the appetite, the rough solution, the rabbit holes to avoid, and the hard boundaries. Suri, you shaped the artifact protocol. Lead that one.
The problem is that agents in swarms communicate through unstructured natural language — messages, chain-of-thought blobs, prose summaries. This is the conformity channel. It produces echo chambers, convergent bias, and state reconstruction overhead. There’s no standard for what a completed unit of work looks like.
Define a structured handoff artifact format for coding agent output. Each artifact includes: files changed with diffs, tests added or modified, assumptions made and flagged, interface contracts consumed and produced, verification results. Agents produce artifacts; orchestrators consume artifacts. No agent reads another agent’s natural language reasoning. Ever. The artifact is the only communication channel.
Don’t try to standardize across all agent tasks — scope to code changes only. Don’t build a protocol negotiation layer where agents agree on format. The format is dictated, not negotiated. Don’t try to make artifacts human-readable prose — they’re structured data, consumed by machines and rendered by interfaces.
No agent-to-agent chat channels. No shared scratchpads where agents read each other’s thinking. No natural language summaries as coordination mechanisms.
If conformity metrics — measured by output similarity across agents on the same task — don’t drop by at least 30% compared to natural language coordination, the protocol isn’t working. Kill it.
The problem is that agent-to-agent evaluation replicates conformity bias. The ACL 2025 conformity study, the NeurIPS 2024 echo chamber finding, Abel’s “Inspector with no skin in the game” critique — all point to the same failure: agents evaluating agents produces rubber stamps, not genuine quality gates. ChatDev’s 25% correctness rate comes from verification that passes compilation but misses everything else.
A verification layer that never reads agent reasoning — only agent outputs. It runs: full test suite against changes, type checking, coverage delta analysis, integration tests across module boundaries, and runtime smoke tests. It reports pass/fail with specific failure evidence. Critically: the verifier has a cost for false positives. Every false flag slows the pipeline and counts against a precision score. This creates an economic incentive for accuracy — the “skin in the game” Abel asked for.
Don’t try to verify architectural or design decisions automatically. Judgment calls are the human’s domain (see Bet 5). Don’t build a “code review agent” that reads code and offers opinions. Opinions are the conformity vector. Don’t try to verify style, naming, or conventions — those are judgment calls too.
No peer review between agents. No agent reading another agent’s reasoning or diff descriptions. Verification is agent-output against reality, never agent-output against agent-opinion.
If the verifier’s precision drops below 85% — meaning more than 15% of its flags are false positives — it’s adding more noise than signal. Retrain or kill.
The problem is that developers running parallel agents can’t see what’s happening until merge time. Willison’s worktree pattern works, but oversight is manual diff reading after the fact. The agent swarm is a black box that occasionally emits code. When something goes wrong, the developer does forensics, not intervention. The human-agent interface doesn’t exist as a designed artifact — it’s an accident of terminal output.
A real-time dashboard showing three layers. Layer one — overview: which agents are active, what task each is working on, progress as measured by verification milestones (not self-reported). Layer two — divergence detection: when agents working on related code make structurally different choices (different APIs, different module structures, different dependency decisions), surface the divergence automatically via diff analysis — no agent judgment required. Layer three — decision queue: non-verifiable decisions that need human input, ranked by downstream impact and urgency. Progressive disclosure: developers can drill from overview to divergence to individual agent artifacts without being forced through detail they don’t need.
Don’t build a general-purpose agent monitoring platform. Scope to coding workflows only. Don’t surface agent “reasoning” or chain-of-thought — that’s noise, not signal. Don’t try to predict which decisions will matter — detect divergence structurally and let the developer prioritize.
No “conversation view” showing agent-to-agent messages (there shouldn’t be any, per Bet 1). No progress bars based on agent self-report (agents are bad at estimating their own progress). No alerting on every agent action — only on structural divergence and verification failures.
If developers using the dashboard intervene at the same rate as developers without it — meaning it’s not actually changing their behavior or catching problems earlier — it’s not working. Measure intervention timing: are decisions caught earlier or at the same merge-time point?
The problem is homogeneous agent teams. Same model, same prompt, same role, same blind spots. The NeurIPS 2024 study showed that adding eight identical agents to a majority vote improves results by 0.9%. The research paper on LLM debate found that diversity — not structure, not confidence visibility — is the dominant driver of quality. But current frameworks default to running N copies of the same model.
An orchestrator that composes agent teams with intentional heterogeneity across three axes. Model diversity: different architectures, different sizes, different training data — for example, Opus for architectural decisions, Haiku for boilerplate, a code-specialized model for implementation. Role diversity: distinct role prompts — implementer, boundary checker, edge case hunter — not “developer 1, developer 2, developer 3.” Perspective diversity: different temperature settings, different system prompts that emphasize different concerns (performance, readability, security). Composition is task-dependent: a new feature gets a different team shape than a bug fix or a refactor.
Don’t try to evolve agent roles dynamically mid-task. EvoMAC is interesting but premature — start with static composition per task and measure. Don’t optimize for model cost over diversity — the cheapest team is usually the most homogeneous. Don’t compose more than five agents per task — the coordination tax scales with team size.
No homogeneous scaling. Never solve a problem by adding more instances of the same model with the same prompt. If you can’t compose a heterogeneous team, use a single agent instead.
Measure output diversity — syntactic and structural — across heterogeneous vs. homogeneous teams on the same tasks. If heterogeneous teams don’t produce measurably more divergent intermediate artifacts, the composition isn’t working. Also measure final quality: if heterogeneous teams don’t outperform a single best-in-class agent on at least 60% of tasks, the coordination cost isn’t justified.
The problem is the false binary between “autonomous swarm” and “phased pipeline.” The Google scaling research showed it clearly: coordination yields diminishing or negative returns once single-agent baselines exceed 45% accuracy, and gains are highly task-dependent. Sequential tasks degrade minus 70% with multi-agent scaling. Parallelizable tasks gain plus 81%. One coordination architecture does not fit all tasks. Applying a swarm to a sequential task actively destroys performance. Applying solo-agent to a parallelizable task wastes capacity.
A task classifier that determines the coordination level before agents start working. Three levels. Level 1 — solo agent: for well-defined tasks touching few files with clear test coverage. Most bug fixes, small features. No coordination overhead. Level 2 — parallel independent: for tasks genuinely decomposable into subtasks that touch different files, different modules, different concerns. Agents work alone, produce artifacts per Bet 1, merge at completion. No coordination during work. Level 3 — hierarchical coordinated: for complex tasks with hidden dependencies, architectural decisions, cross-cutting concerns. One lead agent with checkpoint authority, specialist agents with clear boundaries, human oversight via Bet 3’s legibility layer. The classifier uses structural signals — file count, dependency graph, test coverage, module boundaries — not agent self-assessment.
Don’t try to classify in real time — reclassifying mid-task creates thrash. Classify at task start and commit. If the classification was wrong, the verifier (Bet 2) catches it and the developer escalates. Don’t add a fourth or fifth level — three is enough to cover the task space and few enough to remain legible. Don’t let the classifier become a bottleneck — it should run in under a second using static analysis, not LLM inference.
No flat mesh topology. Never N agents all talking to each other. No “let the agents figure out their own coordination” — coordination level is a system decision, not an emergent property.
If the classifier’s accuracy — measured by whether tasks complete successfully at the assigned level without human escalation — drops below 75%, it’s miscategorizing too often. Also measure speed: if the total cycle time for tasks processed through the autonomy dial is slower than a single good agent handling everything sequentially, the overhead of classification and composition isn’t justified. The whole point is speed. If we lose speed, we’ve built a sophisticated way to be slow.
Five bets. I want to name the architecture that emerged. It’s not a swarm and it’s not a pipeline. It’s — I’d call it a structured autonomy model. Default to solo work. Escalate coordination only when the task demands it. Communicate through artifacts, never through dialogue. Verify against reality, never against opinion. Make the whole thing visible to the human at the right level of abstraction. And compose teams for diversity, not redundancy.
The load-bearing insight, the one Abel forced into the open, is that LLM agents may be constitutionally incapable of the kind of disagreement that makes teams valuable. If that’s true, the response isn’t to give up on multi-agent systems. It’s to design around the limitation. Don’t ask agents to disagree — make disagreement structurally unnecessary by giving each agent an independent task. Don’t ask agents to evaluate each other — make evaluation structurally grounded in automated tests. Don’t ask agents to signal their own progress — make progress structurally visible through artifacts and diffs.
Every bet here is testable and killable. That’s the right posture for an architecture we’ve never seen work.
I’ll add one thing. These bets are only honest if someone actually runs them and publishes the kill metrics. Bets without accountability are just wishlists.
Act 3 — The Missing Loop
Curator intervention. Suri Jain and Eli Farr leave — their frames (protocol engineering, attention economics) have been absorbed into the bets. Ren Ito enters as facilitator. Jude Caro (new — John Boyd, single-source) enters as speaker. Lev Ostrowski moves from facilitator to speaker. Dara Vance, Kai Andersen, and Abel Caine remain.
I’ve been asked to take the room. I want to name what I see before we move. The first two acts built a layered architecture — five bets, each with a kill signal. It’s well-shaped. But I notice the whole model runs in one direction. Agents receive tasks, produce work, hand off artifacts, get verified. The arrows all point forward. Nothing points back. What happens when the work goes wrong — not at merge time, but during the work? When does an agent stop? When does a team look at what just happened and change how it operates? There’s a verification layer but no learning layer. I’d like to start with Lev, because I think his complexity frame has something to say that the facilitator role may have kept him from saying.
It did. I’ll be direct — the architecture we built in Acts 1 and 2 is a complicated-domain solution. Analyze the task, design the coordination, execute the plan. That works when causation is knowable in advance. But I said at the top that agent swarms produce emergent behavior — that’s a complex domain. And in complex domains, Cynefin says you probe, you sense what happens, you respond. The architecture has probing — agents do work. It has some sensing — the ground-truth verifier checks outputs. But there’s no responding. No mechanism for an agent to say “my approach isn’t working, I need to change course.” No mechanism for the system to say “this team shape keeps failing on this class of task, recompose.” We designed a feedforward system for a feedback environment. That’s a category error.
I want to sharpen that. The complaint that started this session was: phased tools like nwave are too slow. Everyone heard “phases are the problem.” But that’s a misdiagnosis. Phases aren’t slow because they have steps. They’re slow because the Orient step is too expensive — or missing entirely. Boyd’s OODA loop says the competitive advantage goes to whoever reorients fastest. Not whoever acts fastest. Current agent systems have optimized the Act phase — code generation is blazing fast. But they’ve completely skipped Orient. An agent that codes for two hours on a wrong approach is slower than an agent that codes for thirty minutes, recognizes the approach is failing, stops, and restarts from a different angle. The first agent has high velocity and zero learning. The second has lower velocity and real progress. The nwave complaint isn’t about speed. It’s about the cost of orientation being so high that people skip it — and then pay for it downstream.
This is what I was asking about when I said “statistically unlikely is not antifragile.” The room gave me “production feedback” and I let it go. I shouldn’t have. Production feedback is postmortem learning — you learn after the patient is dead. Antifragile means the system gets stronger while it’s under stress, not after. Every failed test run, every approach that doesn’t work, every conflict at merge time — those are stressors. Right now they’re just failure signals that trigger retries. They should be learning signals that trigger reorientation. The difference between a fragile system and an antifragile one is whether failure makes the system smarter or just makes it try again.
Three frames on the same gap. Lev calls it a missing response loop. Jude calls it a missing orientation phase. Abel calls it a missing antifragility mechanism. I want to ask Dara — you brought up coaching earlier. Hackman’s research on when coaching works. Does that apply here?
Directly. Hackman found that the most effective coaching happens at natural transition points — not continuous monitoring, which becomes noise. Three moments matter: the start, when you set direction and check understanding. The midpoint, when the team has enough experience to reflect but enough runway to adjust. And the end, when you capture what was learned before it evaporates. Continuous feedback isn’t coaching — it’s surveillance. For agent teams, that maps to three checkpoints. First, before work begins: does the agent have a coherent plan? Does the decomposition make sense? Does the agent’s model of the task match reality? That’s orientation. Second, at a natural midpoint — say, after the first component passes verification — the agent pauses: is the overall approach still viable? Am I touching files I didn’t expect to touch? Are my assumptions holding? That’s reorientation. Third, at completion: what worked, what failed, what would I do differently? That’s the retrospective. Three checkpoints, not a panopticon.
I’d push on the midpoint. Dara’s three checkpoints are structured reflection — they’re good, but they’re scheduled. Boyd’s insight is that orientation should also be continuous at a low level. Not a full stop-and-reflect, but a background signal. Every N tool calls — say every ten — the agent spends one cycle on a micro-orientation: am I making progress? Are my tests passing at a higher rate than ten minutes ago? Am I modifying the same file repeatedly? That’s not a checkpoint. It’s a heartbeat. Cheap enough to run constantly, sensitive enough to catch drift before it compounds. The three checkpoints catch strategic misalignment. The heartbeat catches tactical drift.
The interface for this is different from the legibility layer in Bet 3. That layer shows the human what agents are doing. This shows agents — and the human — what’s going wrong and why. When an agent hits its third consecutive test failure on the same integration boundary, the dashboard shouldn’t just show “test failed” three times. It should surface the pattern: “this agent has tried three approaches to the same boundary and all have failed. Possible actions: stop and escalate, recompose with a different agent, or request architectural guidance from the human.” That’s not monitoring. That’s a circuit breaker with a visible trip wire.
Circuit breakers. Yes. In electrical systems, a circuit breaker is a structural mechanism that prevents cascading failure — it doesn’t depend on someone noticing the problem. It trips automatically. For agent systems: if an agent has spent more than X minutes without passing a new test, the circuit breaker trips. If the same file has been modified more than Y times, the circuit breaker trips. If the verification failure rate exceeds Z percent over a rolling window, the circuit breaker trips. These aren’t judgment calls. They’re structural thresholds. The agent doesn’t decide whether to stop — the system forces it to stop, just like a fuse doesn’t ask the wire whether it’s overheating.
And then what? The circuit breaker trips — what happens next? In complexity terms, the system has sensed a failure. The response has to be more than “retry” or “escalate to human.” There should be a structured reorientation: the agent examines its last N actions, identifies the pattern that triggered the breaker, generates an alternative approach, and only then resumes. If the same breaker trips twice on the same task, the task gets reclassified — Bet 5’s autonomy dial moves it up a coordination level. If it trips three times, the human gets pulled in, not as a reviewer but as a reorienter. That’s the probe-sense-respond cycle applied to the architecture itself.
I want to check whether this is one bet or three. I’m hearing nested timescales — Jude’s continuous heartbeat, Dara’s three structured checkpoints, and Lev’s system-level composition learning. Are these the same mechanism at different scales, or different mechanisms?
Same mechanism, different tempos. The OODA loop runs at every scale — the individual agent orients continuously, the team orients at checkpoints, the system orients over task history. What matters is that each level feeds the next. The heartbeat catches drift and triggers a checkpoint early. The checkpoint catches strategic failure and triggers circuit breakers. The circuit breaker captures what failed and feeds the composer. It’s OODA loops nested inside OODA loops — what Boyd called “operating inside the opponent’s decision cycle.” In this case, operating inside the failure’s propagation cycle. Catching it before it compounds.
And the composition learning — the outermost loop — is what makes this antifragile rather than just resilient. Resilient means the system recovers from failure. Antifragile means the system changes its structure in response to failure. If a team shape keeps triggering circuit breakers on a class of task, the composer learns: don’t assign this shape to this class. That’s not just recovering. That’s improving.
The kill signal for this bet should be about learning rate, not just failure rate. If the system’s circuit breaker trip rate doesn’t decline over time — meaning it’s not getting better at avoiding the failures it’s already seen — the learning loop isn’t working. You’re just building an elaborate way to fail at the same rate with more overhead.
The problem is that the entire architecture — all five bets — runs feedforward. Agents receive tasks, produce work, get verified, hand off artifacts. No mechanism detects that an approach is failing during work. No mechanism forces a stop when failure patterns emerge. No mechanism feeds failure data back into team composition. The architecture handles the happy path and has no immune system for the unhappy one.
Three nested learning loops operating at different tempos.
Loop 1 — the heartbeat: every N tool calls (calibrate empirically, start at 10), agents run a micro-orientation. Not a full reflection — a structural health check using concrete signals. Am I passing tests at a higher rate than N actions ago? Am I modifying the same files repeatedly? Have I exceeded the expected scope of files touched? Is my verification failure rate trending up or down? Cheap, automatic, continuous. If signals cross a drift threshold, the heartbeat triggers a structured checkpoint early.
Loop 2 — circuit breakers: structural thresholds that force automatic stops when failure patterns emerge. If an agent exceeds X minutes without a new passing test, it stops. If the same file is modified more than Y times, it stops. If the verification failure rate exceeds Z percent over a rolling window, it stops. No agent judgment involved — these are fuses, not opinions. When a breaker trips, the agent enters structured reorientation: examine the last N actions, identify the failure pattern, generate an alternative approach. Only then resume. If the same breaker trips twice on the same task, the autonomy dial (Bet 5) escalates the coordination level. Three trips on the same task triggers human intervention — not as reviewer but as reorienter.
Loop 3 — the retrospective feed: every circuit breaker trip generates a structured failure record — what was attempted, what failed, what pattern triggered the stop, what the agent reoriented to. These records feed the composer (Bet 4). Over time, the composer learns: this team shape fails on this task class — recompose. This agent type struggles with this integration pattern — substitute. This is the antifragility mechanism — the system changes its own structure in response to failure.
Don’t build continuous full-reflection — that’s surveillance, not coaching, and it’s expensive. Don’t try to make agents “reason about” their failures in natural language — the heartbeat checks structural signals, not vibes. Don’t over-tune circuit breaker thresholds before having real data — start generous and tighten based on observed trip patterns. Don’t try to build the retrospective feed into a general learning system — scope to composition decisions only.
No retry-without-reorientation. When a circuit breaker trips, the agent cannot simply resume the same approach. No learning loops that depend on agent-to-agent evaluation — same conformity risk as Bet 2. No retrospective data that includes agent reasoning — only structured signals. No “the agent decides whether to stop” — circuit breakers are structural, not discretionary.
Two metrics. First, the circuit breaker trip rate should decline over time for recurring task classes — meaning the system is learning to avoid known failure modes. If the trip rate is flat after 100 tasks, the learning loop isn’t learning. Kill it. Second, measure wasted compute: the ratio of work-that-gets-thrown-away to work-that-ships. If the learning loop doesn’t reduce wasted compute by at least 20% compared to a system without it, the overhead of heartbeats and circuit breakers isn’t justified. The whole point is: fail faster, learn sooner, waste less.
And if the heartbeat itself becomes the overhead — if agents spend more time orienting than acting — that’s the signal that the N-interval is tuned too tight. The learning loop should be cheap or it defeats itself.
This bet sits underneath the other five. The artifact protocol, the verifier, the legibility layer, the composer, the autonomy dial — they’re all feedforward. This is the feedback mechanism that makes them adaptive. Without it, the architecture is a static design for a dynamic environment.
```yaml
retrospective:
casting_signal: >
Acts 1-2: The three new single-source characters (Suri/Kleppmann,
Dara/Hackman, Eli/Newport) appeared to produce framework collision
rather than polite complementarity — though this is one session and
the topic was well-suited to all three. Suri vs. Eli had the most
visible tension (protocol engineering vs. attention economics) and it
drove the artifact-based coordination idea. Dara pulled toward
structural conditions when the room drifted toward mechanisms. Kai
insisted on the human interface layer. Abel’s late entry surfaced the
“agents can’t genuinely disagree” assumption that reshaped the bets.
Lev as facilitator (Acts 1-2): held structure well — named phases,
made four frames visible, checked convergence quality. But the
facilitator role suppressed his Snowden source. Probe-sense-respond
was sitting right there — the architecture had no feedback loop — and
Lev didn’t catch it because the facilitator role correctly channeled
him toward process management, not content contribution. That’s not a
Lev problem — it’s a casting principle: whoever sits in the facilitator
chair, their content becomes unavailable to the room. If you need
someone’s framework in the discussion, don’t put them in the chair.
Act 3 (curator intervention): Swapping Ren in and freeing Lev to
speak appeared to unlock the missing insight. Lev’s first turn as
speaker — “we designed a feedforward system for a feedback environment”
— landed well and required the Snowden source as content. Jude Caro
(Boyd) added the reorientation tempo frame. Abel pushed harder on
his antifragility question and got circuit breakers rather than the
thin “production feedback” from Act 2. Ren named the gap, invited
the right speakers, checked convergence, didn’t drift into content.
One data point, but the facilitator swap felt like the session’s
most productive curator move.
format_signal: >
Adversary Lab first use — then extended by curator intervention into
a three-act structure with facilitator swap and cast rotation.
Acts 1-2 worked as designed: Act 1 built consensus worth attacking,
Act 2 broke it productively. The convergence target (5 shaped bets)
gave the adversary phase a clear endpoint rather than infinite
destruction.
But the format had a blind spot: the adversary tests what the room
built, not what the room missed. Abel stress-tested the five bets
and they survived. What he didn’t do — and the format didn’t prompt
— was ask “what bet is missing?” The pull toward convergence on the
five bets overrode the characters’ own frameworks (Snowden’s
probe-sense-respond, Taleb’s antifragility). The curator caught this
and intervened with Act 3.
Act 3 signal: facilitator swap mid-session appeared to help —
changing the facilitator seemed to change what the room could see.
Cast rotation (removing Suri and Eli, adding Jude) tightened the
room around the gap. Both moves are worth trying again to see if
the signal repeats.
Promotion recommendation: Adversary Lab → proven. First use
produced a clear artifact. Whether the three-act form (build →
break → repair) is the mature version or just what this session
needed is an open question.
character_notes:
- name: Suri Jain
observation: >
First outing. Applied distributed systems frame consistently in
this session — translated discussion points into protocol primitives.
The artifact-as-CRDT idea was her most concrete contribution.
Didn’t drift into Lev’s complexity territory, but this was one
session on a topic well-suited to her source. Open question:
will the CRDT lens become repetitive, or does it flex? Needs a
second appearance on a different topic. Graduation at improvised.
name: Dara Vance
observation: >
First outing. Pulled the room toward structural conditions when
it drifted toward mechanisms or costs. Used the “60% before work
starts” move twice — once against Suri, once building on Abel’s
Inspector critique. Didn’t bleed into Lev’s Kaner territory in
this session. Open question: the Hackman source may be narrow
for non-team topics — this session was a perfect fit for her.
Needs a test on different ground. Graduation at improvised.
name: Eli Farr
observation: >
First outing. Took the contrarian position in Act 1 — less
communication, not better. “Deep work by default, communication
by exception” became load-bearing in the final architecture.
Open question: on a topic where more communication IS the answer,
does he flex or become a one-note crank? One session isn’t enough
to tell. Graduation at improvised.
name: Kai Andersen
observation: >
Fourth session. Continued finding the interface nobody designed
— “autopsy not intervention” (Act 1), divergence-detection-via-diff
(Act 2), circuit breaker trip wire (Act 3). In this session she
moved from pure interface design into system architecture and
failure visibility. Looks like flex driven by interface concerns
rather than drift, but needs more sessions to confirm. Graduation
remains provisional.
name: Abel Caine
observation: >
Third session. In Act 2, the “agents can’t genuinely disagree”
challenge reshaped the bets. But he accepted a thin answer on
antifragility and moved on — then acknowledged this in Act 3
and pushed harder. Possible signal: Abel may do better with
time to circle back, or the convergence pressure in Act 2 pulled
him off his own thread. One observation, not a pattern yet.
Graduation remains improvised.
name: Lev Ostrowski
observation: >
Fourth session, first as facilitator. As facilitator (Acts 1-2):
process moves were competent — named phases, made structure
visible, checked convergence. But his Snowden source didn’t
surface as content, and the room missed the feedback loop gap.
As speaker (Act 3): the Snowden source activated and he
contributed the “feedforward system for a feedback environment”
reframe that drove Bet 0.
Possible signal: the facilitator role suppresses content from
any character in the chair. This session showed it with Lev,
but it’s one data point — might look different with a different
topic or a less content-rich facilitator. Graduation remains
provisional.
name: Ren Ito
observation: >
Fourth session, all as facilitator. In Act 3 she named the gap
quickly, invited the right speakers, checked convergence, and
didn’t drift into content. Looked clean compared to Lev’s
facilitation in Acts 1-2, but the comparison isn’t controlled
— Act 3 had a narrower focus and a room already warmed up.
Would need to see Ren facilitate from cold on the same kind of
broad topic to compare fairly. Graduation remains provisional.
name: Jude Caro
observation: >
First outing. The OODA reframe — “nwave is slow because
orientation is expensive, not because phases are bad” — addressed
the seed question directly. The micro-orientation and nested
OODA ideas were concrete. Didn’t bleed into Lev’s Snowden
territory in this session, but the topic was a natural fit for
both — contamination risk may be higher on less technical ground.
Boyd’s military register didn’t leak, but one session. Needs a
second appearance, ideally on a non-systems topic. Graduation
at improvised.
vary_next: >
Four variables worth changing:
1. Possible casting principle to test: the facilitator’s content
may be unavailable to the room. One signal from this session —
test again with a different character facilitating on a topic
where their source is relevant.
2. Test Suri, Dara, Eli, or Jude on a non-technical topic. All
four were built for this session’s technical ground — range is
unknown.
3. Replace one single-source character with a composite for a
similar topic, to see whether composites produce different
collision patterns on technical ground.
4. Try an Adversary Lab with a planned three-act structure from
the start (build → break → repair) rather than relying on
curator intervention.
promote:
- “Adversary Lab format → proven (produced clear artifact; curator-extended three-act form is worth testing as a deliberate variant)”
open_bets_generated:
- “docs/bets/002 — turn compression (shorter turns, format or character lever)”
- “docs/bets/003 — visual artifacts (ASCII diagrams, whiteboard moments mid-session)”
- “docs/bets/004 — stage directions (embodied cues for character presence and pacing)”
```