Budgeting a context window for a long task

What it is

The context window is everything the model sees on one call: system prompt, tool definitions, the conversation so far including tool results, and the tokens it generates. The vendor documentation on context windows describes how these components count toward the limit and points to server-side compaction as the strategy for conversations that approach it. Two properties make the window a budget rather than a container. The cited paper on long contexts reports that performance is often highest when relevant information occurs at the beginning or end of the input and degrades when it sits in the middle. The cited Anthropic engineering post names the same effect context rot: as tokens in the window increase, the model's ability to recall information from it decreases.

Why it matters

A long task (a multi-file refactor, a research report, a migration) produces far more text than it needs at any one moment. Without a budget, the window fills with old tool output, the instruction given early is now in the middle, and the answer is cut off by the output limit. Each of these is a silent failure: nothing errors, the result is just worse.

How to apply

• Split the budget into four parts and give each a size: fixed (system prompt, tool definitions), task material (the specification, the files in scope), working set (recent tool results, at most a few steps deep), and reserve (the longest answer the task can require plus room for one more tool round).
• Put the specification and the acceptance criteria at the start; restate the current sub-goal at the end of each turn, so the two ends carry what matters.
• Keep the plan, progress and decisions in a scratchpad file the agent rewrites, and reload it after every compaction; compaction summaries are lossy and should not be the only copy.
• Set a compaction point well below the limit and compact by intent (what was decided, what remains) rather than by chronology.
• Track usage from the provider's token counts each call, and stop to hand off or split the task when the working set no longer fits after compaction.
• Prefer several focused tasks with fresh contexts to one context that carries everything.

Pitfalls

Filling the window because it is large. Compacting away the acceptance criteria. Setting the output limit from habit and truncating the final answer. Trusting a compaction summary that dropped an unknown, which then reads as settled.