The Response Format Must Be a Contract
We controlled the input and the behavior - what is left is to pin down the output. If a program reads the result, don't ask it to "return roughly this JSON". Use structured output or a tool schema, and check the meaning in domain code.
type SupportDecision = {
intent: "faq" | "account" | "refund" | "technical";
answerability: "answer" | "clarify" | "escalate";
answer: string | null;
clarification_question: string | null;
source_ids: string[];
proposed_action: null | {
type: "preview_refund" | "create_ticket";
reason: string;
};
};Conformance to the schema itself is only the first of three checks, and each is done by its own layer.
- Syntax - are the required fields present and the enums valid. The schema or SDK answers this.
- Semantics - do this source_id, order_id, currency actually exist. Domain code answers this.
- Authority - does the actor have the right to see and change the entity. Backend policy answers this.
The separation matters because a valid form guarantees nothing about substance.
A schema doesn't prove truth. A string that looks like an ID can still be invented, and a valid enum doesn't prove correct classification. Structured output simplifies integration but doesn't replace evals and data checks.
Finally, a contract is needed for human-facing text too, not only for a machine answer. Set the length, order, language, required links, allowed terminology, and the next step. Instead of a vague "be brief" write something verifiable: "no more than four sentences and one question at the end".
The model's answer passed schema validation. What does that prove?