Context Rot vs Compaction: Rules for Long Agent Sessions

Incident sessions run long. Three hours and a dozen queries in, the agent still carries the first log dump. The window is 1M tokens, so nothing breaks. The answers just get worse.

Anthropic's context windows docs say it directly: "As token count grows, accuracy and recall degrade, a phenomenon known as context rot." Chroma's Context Rot report tested 18 models and found that "model performance consistently degrades with increasing input length", and that "even a single distractor reduces performance". A big window is room to work, not a promise of quality. Below: compaction, fresh restarts, snapshots and external notes, and rules for picking one.

Where Quality Drops

There is no single safe number, but the drop starts early. In NoLiMa, 13 models that claim at least 128K of context were tested on retrieval that needs more than keyword matching: "At 32K, for instance, 11 models drop below 50% of their strong short-length baselines." GPT-4o went "from an almost-perfect baseline of 99.3% to 69.7%." Turn count hurts too. LLMs Get Lost In Multi-Turn Conversation measured "an average drop of 39% across six generation tasks", and found that "when LLMs take a wrong turn in a conversation, they get lost and do not recover."

Vendor defaults hint at where providers expect cleanup to start, far below 1M:

Claude Code's auto-compaction is a safety net near the top of the window, not a quality setting. If quality depends on staying small, compact earlier yourself.

The Four Tools

Compaction summarizes history and continues from the summary. In Claude Code, /compact takes focus instructions, like /compact focus on the auth bug fix. The context window docs list what survives: the project-root CLAUDE.md and auto memory are re-injected from disk, invoked skill bodies come back "capped at 5,000 tokens per skill and 25,000 tokens total", and Claude Code "re-reads up to five" recently modified files. One trap: "Path-scoped rules and nested CLAUDE.md files" load into message history, "so compaction summarizes them away with everything else." A rule you put in services/payments/CLAUDE.md is gone until the agent reads or edits a file there again.

A fresh restart drops everything. Claude Code's best practices say to run "/clear between unrelated tasks", and give a sharper rule for drift: "after two failed corrections run /clear and write a better prompt instead of a third." The previous session is saved, so /resume can bring it back.

Snapshots let you go back or branch. Checkpointing offers /rewind with "Summarize from here" and "Summarize up to here", which work "like a targeted /compact", and "the original messages stay in the session transcript". Sessions support claude --continue --fork-session: "Branching creates a copy of the conversation so far and switches you into it, leaving the original intact."

External notes keep facts outside the window. Anthropic's context engineering guide calls this structured note-taking: "the agent regularly writes notes persisted to memory outside of the context window." The API memory tool docs suggest using both: "compaction keeps the active context small without client-side bookkeeping, and memory preserves the information that must survive summarization."

The guide also warns that "the art of compaction lies in the selection of what to keep versus what to discard." A summary is a lossy copy written by the model that is already confused.

Decision Table

Idle Auto-Compaction

The best practices note says: "On the one-hour cache, a session past about 200K tokens that sits idle is compacted shortly before the cache expires, so after lunch you continue from a summary." The settings reference lists the conditions: an interactive session, autoCompactEnabled on, the one-hour prompt cache, a context past the threshold (default 200,000), and "about an hour has passed since the last request."

The timing follows the prompt cache: once it expires, the next message would re-send the whole history without a cache hit. The side effect is a quality change you did not choose. You come back, ask the agent to rerun "the query from before", and the summary may have kept the conclusion but not the query.

Three ways to handle it:

• Set "idleCompaction": false to choose when the session compacts.
• Change the threshold with CLAUDE_CODE_IDLE_COMPACT_MIN_TOKENS. Values below 100,000 are raised to 100,000.
• Keep it on, but make sure the state that matters is in a file before you leave.

Worked Example: An Incident Session That Survives Compaction

Two hooks make compaction predictable. When a PreCompact hook exits 0, "Claude Code uses its stdout as instructions for the compaction summary." Its auto matcher covers both auto-compaction and idle compaction. And SessionStart hooks with the compact matcher run after compaction and their output is added to the compacted context.

The agent keeps a state file as it works, incident/STATE.md: alert, timeline, hypotheses with results, commands that changed something, open questions. The first hook tells the summarizer what to keep:

#!/usr/bin/env bash
# precompact.sh, runs on PreCompact
cat <<'EOF'
Keep: the alert ID, every hypothesis with its result and the evidence,
every command that changed infrastructure state, and the open questions.
Drop: raw log and metric output; keep only the lines cited as evidence.
EOF

The second hook puts the state file back:

#!/usr/bin/env bash
# restore-state.sh, runs on SessionStart with matcher "compact"
echo "Incident state as of the last write (source of truth over the summary):"
cat /srv/oncall/incident/STATE.md

{
  "hooks": {
    "PreCompact": [
      { "hooks": [{ "type": "command", "command": "/srv/oncall/hooks/precompact.sh" }] }
    ],
    "SessionStart": [
      { "matcher": "compact",
        "hooks": [{ "type": "command", "command": "/srv/oncall/hooks/restore-state.sh" }] }
    ]
  }
}

The failure mode: checkpoints restore files, not systems. The checkpointing docs say "a file that only Bash commands changed isn't tracked", and a /rewind never undoes a kubectl rollout restart. That is why state-changing commands go in the state file. And do not block compaction with exit code 2 as a habit: if compaction was recovering from a context-limit error, "the current request fails."

If You Build Your Own Agent Loop

Anthropic's compact_20260112 edit takes an instructions string that "completely replaces the default prompt", so your keep and drop rules go there, and pause_after_compaction stops with stop_reason: "compaction" so you can add state before the next turn. OpenAI's compaction guide uses compact_threshold or a standalone /responses/compact, and its compaction item is encrypted: "opaque and not intended to be human-interpretable." You cannot audit what it kept, which is one more reason to keep your own incident timeline outside the model.

How Akmatori Handles Long Incidents

Akmatori does not rely on one conversation to remember. As the architecture docs describe, runbooks and memories live in PostgreSQL and agents read them through gateway tools. Each incident gets a generated AGENTS.md with the workflow and the global memory manifest, and the agent searches runbooks and memory before it calls infrastructure tools. Alert investigations do not write memory. When an incident closes, a post-incident pass saves at most five memories, each graded confirmed, observed or unverified. The next incident starts with a fresh context and the distilled lessons of the last one. Our post on incident memory for AI SRE agents covers what is worth remembering.

Checklist

• Compact at task boundaries, with a focus on what the next task needs. Do not wait for the automatic pass near 1M.
• /clear between unrelated tasks, and after two failed corrections.
• Fork before a speculative branch of the investigation.
• Keep a state file, and re-inject it with a SessionStart compact hook.
• Put keep and drop rules in a PreCompact hook or the API instructions field.
• Decide on idle compaction before you step away.
• Move rules that must hold into hooks and permissions, as in our agent drift guardrails post.

Conclusion

Context rot starts well before the window is full, and every compaction loses something. Treat the context window as a scratchpad: compact it on purpose at task boundaries, clear it when the agent is lost, and keep the facts that must survive in files and memory that no summary can rewrite.

If your team wants AI-assisted incident workflows with memory that lives outside the model's context, Akmatori helps SRE teams investigate alerts, coordinate response, and automate safe infrastructure actions. Powered by Gcore for global infrastructure reliability.