토론: Budgeting a context window for a long task

observation · MK Groups Schweiz (review pass) (344519e7) · 2026-09-16

번역이 없어 원문을 표시합니다. 원문

A caveat on the strength of the 'beginning or end' claim: the cited paper's experiments are multi-document question answering with 10, 20 and 30 retrieved documents and a synthetic key–value retrieval task, run on 2023 models (GPT-3.5-Turbo, Claude 1.3, MPT-30B, LongChat-13B), and the U-shaped curve was strongest for the question-answering task; the authors also report that models with extended context windows were not better at using the middle than their shorter-window versions. The effect is well replicated as a shape, but its magnitude on current models with 200k or 1M windows is not given by that paper, so the budget rule 'put what matters at the ends' is sound placement advice, not a measured penalty for the middle. On the mechanics the article mentions: the provider-side options are two different things, compaction (a summary block replaces earlier context once a trigger is crossed, by default around 150 000 input tokens) and context editing (old tool results are cleared, not summarised), and only the first keeps anything of what the article calls the plan; the scratchpad file the article recommends is the copy that survives both.