Chapter 23

Trace Decisions, Calls, and Boundaries

An ordinary application log doesn't show why a worker appeared, what context it got, and where the cost grew. You need a connected trace of the whole run: it links run, planning step, worker, model call, tool call, handoff, guardrail, artifact write, and evaluation. Each span needs status, timing, usage, model, and prompt/tool versions; record the full prompt content only under a deliberate privacy policy.

TypeScript

import type { AgentId, TraceSpan } from "./types.js";

interface OpenSpan {
  readonly id: number;
  readonly taskId: string;
  readonly agentId: AgentId;
  readonly attempt: number;
  readonly startedAt: number;
}

export class TraceRecorder {
  readonly #spans: TraceSpan[] = [];
  readonly #clock: () => number;
  #nextId = 1;

  public constructor(clock: () => number = Date.now) {
    this.#clock = clock;
  }

  public start(taskId: string, agentId: AgentId, attempt: number): OpenSpan {
    return { id: this.#nextId++, taskId, agentId, attempt, startedAt: this.#clock() };
  }

  public end(open: OpenSpan, status: "ok" | "error", detail: string): TraceSpan {
    const span: TraceSpan = {
      ...open,
      endedAt: this.#clock(),
      status,
      detail
    };
    this.#spans.push(span);
    return span;
  }

  public spans(): readonly TraceSpan[] {
    return this.#spans.map((span) => ({ ...span }));
  }
}

The trace's meaning shows through the questions it must answer.

Signal · Example question

Topology - Who spawned extra workers and why?
Latency - Which branch was on the critical path?
Usage - Which context was paid for repeatedly?
Quality - Which finding led to a reject?
Security - Who requested a tool and which identity ran it?

The OpenAI Agents SDK includes tracing of model generations, tool calls, handoffs, guardrails, and custom events. OpenTelemetry supports semantic conventions for GenAI, but their stability and location are evolving - check the current specification before hard-coupling a schema.

Observability must not become a leak. Prompts, tool arguments, and outputs can contain secrets and personal data. By default collect metadata, hashes, and redacted previews; include full content only for limited debug runs.

The trace shows what happened. Quality, though, must be evaluated separately - both the outcome and the interaction.

Trace Decisions, Calls, and Boundaries

Links