Chapter 11

Parallelize a Dynamic Number of Independent Workers

Orchestrator-workers fit breadth-first search and map/reduce tasks, where the number of branches is set by the input's content and the outputs can be combined by an explicit rule. A lead agent or planner first builds a list of independent work items; the control plane checks the fan-out size, duplicates, write sets, and budget, then runs a bounded number of workers; a reducer combines structured outputs, not long conversations.

Plan: set of work items
Validate: limit, dedupe, scope
Fan out: bounded concurrency
Fan in: schema and provenance
Gap check: stop or second round

Anthropic describes a production research system exactly as orchestrator-worker: a lead agent creates specialized subagents that explore directions in parallel, after which a separate process assembles the result and citations. The same material shows an early failure mode: a simple query could spawn dozens of agents with no reasonable budget.

Fan-out limits are therefore mandatory: a maximum of workers per run and per planning step; a deduplication key for identical work items; a concurrency limit separate from the total worker limit; a local token, tool-call, and time budget; a ban on recursive spawn or an explicit depth limit; a gap check that can end the run without a new round. In code the basis for this is a ready set over the DAG.

TypeScript

import type { TaskStatus, WorkItem } from "./types.js";

export function readyTasks(
  tasks: readonly WorkItem[],
  status: Readonly<Record<string, TaskStatus>>
): readonly WorkItem[] {
  return tasks.filter((task) =>
    status[task.id] === "pending" &&
    task.dependsOn.every((dependency) => status[dependency] === "completed")
  );
}

export function createInitialStatus(tasks: readonly WorkItem[]): Record<string, TaskStatus> {
  return Object.fromEntries(tasks.map((task) => [task.id, "pending" as const]));
}

export function hasUnfinishedTasks(status: Readonly<Record<string, TaskStatus>>): boolean {
  return Object.values(status).some((value) => value === "pending" || value === "running");
}

Parallelism saves only wall-clock. It cuts time only for independent branches, while total tokens, calls, and cost usually grow. So in the report keep both the critical path and the total consumption.

The branches are running. When a critic appears among them, it should give a review by evidence, not an endless debate.

Knowledge check

What does running independent workers in parallel save?

Parallelize a Dynamic Number of Independent Workers

Links