Chapter 12

Use Review, Not an Endless Debate

A critic is useful if it has independent evidence and the right to reject a result by a rubric. A free group chat often only multiplies text and correlated errors. The producer-critic pattern looks convincing but easily turns into a loop of mutual agreement: the reviewer must get immutable artifacts, acceptance criteria, and check results, not assess the producer's hidden reasoning and not believe "tests passed" without evidence.

Three modes differ by the source of trust. Self-critique is cheap, but errors and assumptions correlate strongly - the same context. An independent reviewer reads the task contract, artifacts, and evidence in a fresh context, not the executor's history of beliefs. A group chat suits some collaboration patterns but grows the context and requires hard termination. In the companion project the review is precisely evidence-gated.

TypeScript

  private review(context: AgentContext): AgentResult {
    const evidenceKeys = passedEvidence(context);
    const required = ["evidence/tests/result.json", "evidence/security/result.json"];
    const missing = required.filter((key) => !evidenceKeys.includes(key));
    if (!artifactExists(context, "release-notes/notes.md")) {
      missing.push("release-notes/notes.md");
    }
    const passed = missing.length === 0;
    const reviewEvidence: Evidence = {
      name: "independent-review",
      passed,
      details: passed ? "All independent branches supplied evidence" : `Missing: ${missing.join(", ")}`
    };
    if (!passed) {
      throw new Error(reviewEvidence.details);
    }
    return {
      summary: "Independent reviewer accepted the release package",
      artifacts: [{
        key: "report/release.json",
        content: JSON.stringify({ decision: "approve", evidence: evidenceKeys }),
        expectedVersion: 0
      }],
      evidence: [reviewEvidence],
      usage: USAGE.reviewer
    };

AutoGen documents group-chat teams where agents post messages to a shared context in turn, and separately advises starting with a single agent for simple tasks; in the termination documentation conditions are checked after the agents' answers - that's part of the pattern, not an optional improvement. Set the termination for the review loop explicitly: a pass of all mandatory deterministic graders; a maximum of N revision rounds; no new material finding; the local budget exhausted; escalation to a human on a requirement dispute, rather than one more opinion agent.

Don't vote on facts. Three agents that read one wrong source don't create three independent proofs. Count provenance and the diversity of evidence, not the number of identical conclusions.

The review is arranged. But for the role to be checkable at all, define it by a contract, not a profession.

Use Review, Not an Endless Debate

Links