Chapter 16

RAG Returns Evidence, Not a New Prompt

RAG in an agent isn't "mix documents into the prompt" but a tool that returns evidence with provenance. policy_search filters documents by tenant, locale, and validity, each fragment gets a stable source id, and the document text is treated as untrusted data - even if it came from your own database.

TypeScript

const policySearchInput = z.object({
  query: z.string().min(3).max(300),
  locale: z.enum(["ru", "en", "uz"])
}).strict();

async function policySearch(raw: unknown, context: TrustedToolContext) {
  const input = policySearchInput.parse(raw);
  const now = new Date();

  const hits = await retrieval.search({
    query: input.query,
    tenantId: context.tenantId,
    locale: input.locale,
    filter: {
      activeAt: now,
      status: "published"
    },
    limit: 5
  });

  return {
    passages: hits.map(function toPassage(hit) {
      return {
        source_id: hit.sourceId,
        title: hit.title,
        effective_at: hit.effectiveAt,
        excerpt: hit.excerpt
      };
    })
  };
}

Search alone isn't enough: you also need to check that the model cited only what was actually found. Citations are verified after the answer.

TypeScript

function verifyCitations(
  requested: string[],
  retrieved: Set<string>
) {
  const unknown = requested.filter(function isUnknown(id) {
    return !retrieved.has(id);
  });
  if (unknown.length > 0) throw new Error("unknown_citation");
}

Hence an important rule about what to do when there's no evidence.

Answerability gate. If the search didn't return a sufficient active source, the right outcome is a clarification or a handoff. The model doesn't fill in policy with general knowledge: an honest "not found" beats a confident fabrication about the store's rules.

The retrieval itself can be built in different ways, and the choice affects the whole cross-provider design.

Retrieval option · Plus · Limitation

Your own vector or hybrid search - The same corpus and eval for three providers; You're responsible for ingestion, ranking, and ACL
Provider file search - Less infrastructure; Different indexes, filters, citations, and lifecycle
Search as a separate service - A clear boundary, cache, and independent scaling; Extra network and observability

Your own vector or hybrid search gives the same corpus and eval for the three providers at the price of ingestion, ranking, and ACL being on you. Provider file search removes the infrastructure but brings different indexes, filters, and citations for each API. Search as a separate service gives a clear boundary at the price of an extra network hop. For the book portability matters, so the corpus is shared, not provider-specific.

RAG answers for the rules. But the agent also reads a specific order - and that's personal data, and such a read starts with an ownership check.

RAG Returns Evidence, Not a New Prompt

Links