Context Is an Attention Budget
Reasoning works on what it sees, and it sees a limited window. A large context window doesn't mean any volume of data is equally useful: the engineer's job is to pick the minimal set of high-signal tokens for a specific step.
It helps to distinguish what goes into the context at all.
- Stable - system instructions, tool definitions, glossary, a policy summary, the output schema.
- Dynamic - the current request, session state, retrieved chunks, tool results, a temporary plan.
- History - only those messages and events that actually change the decision at the current step.
Most context problems come from the habit of sending everything at once. Each has a working replacement.
- A long conversation. Instead of "send everything every time" - facts in state, a summary with provenance, and the last relevant turns.
- A large knowledge base. Instead of "paste the documents whole" - hybrid retrieval, filters, rerank, and compact chunks.
- Many tools. Instead of "load hundreds of schemas" - a minimal set, namespaces, or deferred tool search.
- A long agent task. Instead of relying on chat history - an explicit plan, checkpoints, artifacts, and resumable state.
- A repeated prefix. Instead of reshuffling sections every time - stable data at the start, for the sake of prompt caching.
Separately, on placing long data. Anthropic and Google advise putting a large corpus before the specific question and phrasing the task itself closer to the end.
<documents>
{{long_context_with_source_metadata}}
</documents>
# Task
Based on the documents above, compare the two policies.
For each difference, state the source_id and the effective date.
If the versions can't be ordered by date, return policy_conflict.This isn't a universal law for every API but a reasonable starting configuration for long documents, to be confirmed on your own evals. And memory isn't the same as truth.
Memory isn't truth. A summary of a past conversation may contain an error. Keep facts separate from interpretations, add provenance, and don't let memory raise a user's privileges.
A summary of a past conversation says the customer was approved for a refund. Can you rely on it?