Skip to content
MiraiTech
AI Lab

Experiment · Agents

Multi-agent debate

Three agents take the roles of proponent, opponent and devil's advocate and argue over several rounds. A judge scores each argument on logic, evidence and novelty, then writes a balanced conclusion. We use it to compare how different models reason.

Type
Agents
Status
Live
Models
Claude Sonnet, GPT-4o, Gemini 2.5 Pro

Results

  • 3 rounds per debate
  • Streams while it runs
  • Models compared side by side

Across 30 debate topics, debated conclusions scored 23% higher on factual accuracy than single-model answers, as judged by 5 human evaluators. A 3-round debate took 45 seconds on average. Streaming kept token-to-screen latency under 200ms. The best conclusions came from mixed-model debates where Claude argued, GPT-4o cited evidence and Gemini played devil's advocate.

What we tested

This experiment tests whether LLMs reason better when they argue. Three agents debate a topic over several structured rounds, and a judge agent scores the arguments and writes a conclusion. We compared the debated conclusions with single-model answers, and compared Claude, GPT-4o and Gemini with each other.

Method

The framework has 3 roles (Proponent, Opponent, Devil's Advocate) and a Judge. Each debate runs 3 rounds: opening statements, rebuttals and closing arguments. The judge scores each argument on logic (0-10), evidence quality (0-10) and novelty (0-5), then writes a balanced conclusion. We ran 30 debate topics across ethics, technology policy and business strategy. Each topic was debated 3 times with different model assignments, to control for bias between model and role.

Findings

  1. 01

    Debate improved factual accuracy by 23%

    When models argued opposing positions, the final conclusion had 23% fewer factual errors than a single model's answer to the same question. The opposing agents challenged unsupported claims.

  2. 02

    Claude scored highest on logic

    Claude Sonnet scored highest on logic (8.2/10 on average) and wrote the best-structured arguments. GPT-4o scored highest on evidence citation (7.8/10). Gemini was strongest on novel perspectives (3.9/5).

  3. 03

    The Devil's Advocate role matters

    Removing the Devil's Advocate and running 2-agent debates lowered conclusion quality by 18%. The contrarian role makes both sides deal with edge cases they would otherwise ignore.

  4. 04

    Three rounds was enough

    We tested 2, 3 and 5 rounds, and returns fell off after round 3. Rounds 4-5 repeated earlier arguments and added 60% to latency without improving the conclusion.

How it is built

A LangGraph state machine runs the debate. Once the topic is set and roles are assigned, round 1 runs the opening statements in parallel and the judge scores them. In round 2, the agents give rebuttals one after another, each with access to the opposing arguments, and the judge scores again. Round 3 is closing arguments, followed by the final synthesis. Every transition streams tokens to the frontend over WebSockets. The judge uses a separate, higher-temperature prompt so it does not anchor on any one agent's framing.

LangGraph runs the debate flow, with explicit state transitions between rounds. Arguments stream to the frontend over WebSockets. Redis stores debate state so a debate can recover from an interruption. Next.js renders the debate UI as the arguments stream in.

  • LangGraph
  • Next.js
  • Redis
  • WebSockets

Problems we hit

Agents conceded too easily

In rebuttals, agents sometimes conceded valid points too quickly and the debate collapsed. We added a "maintain your position" directive to the system prompt and penalized concessions in the judge's scoring rubric.

Streaming 3 models at once

Three concurrent LLM streams at different speeds caused rendering problems. We added a token buffer that batches and interleaves the streams at 100ms intervals so the UI updates smoothly.