TokenEconomizer: Cutting Coding Agent Token Costs

TokenEconomizer is an Apache 2.0 plugin from Microsoft Research, Microsoft Security and UC Berkeley that makes coding agents cheaper in the repositories you use them in every day. It is tempting to describe it as a cache that records an agent's usual patterns and replays them. That description is close enough to spread and wrong enough to mislead. None of the mechanisms its README and paper describe caches model responses or replays them. It watches finished sessions, learns what keeps costing tokens in one repository, and changes what the next session starts with.

Below: the three mechanisms, what an agent update breaks, the measured savings, and when a newer model is the better buy.

What It Actually Does

The paper starts from an observation: agents are benchmarked on isolated tasks, but developers run them again and again on the same project. In that setting, agents show recurring "environment-dependent" token costs. The agent rediscovers the same facts, carries the same unused tools, and falls into the same failing loops. The README names three mechanisms, each one switchable on its own:

Mechanism	Waste it targets	How it acts	Model calls	Default
Micro-skills	Re-learning the repo (python -m pytest where pytest is missing, a build step that must run first)	States a fact at session start once three sessions confirm it, drops it when later sessions contradict it	A fact proposer every 3 sessions; envfacts mode uses none	Off
Tool pruning	Tool definitions sent with every request but never used	Writes a deny list into the agent's own settings after a tool goes unused for 10 prompts over 2 sessions, or for 5 sessions	None	On
Loop guard	A turn stuck on the same failures	A judge model checks the turn; at 0.7 confidence it adds one short redirect, then only tells you how to stop the turn	Judge and redirect writer, each prompt about 1,500 tokens at most	On

Two design choices matter. First, everything goes through what the agent already reads: facts enter the context at session start, and tools are removed through .claude/settings.local.json, .codex/config.toml or .github/copilot/settings.local.json. The agent's model and code stay untouched, hence "black-box cost control". Second, cost is counted as billed, with cache pricing, including the plugin's own model calls. The plugin docs say those calls run through your own CLI (claude -p, codex exec, copilot -p) on your existing login, with Haiku as the default on Claude Code.

The "sleep memory" comparison fits micro-skills only: a periodic review of recent sessions proposes facts, and repeated evidence promotes them. The other two mechanisms are counters and a judge.

The Measured Savings

On 277 SWE-Bench tasks from six repositories, each repository's tasks run in order with nothing learned at the start, the authors report 18.4 to 30.9% lower mean cost across five agent and model combinations. Every one of the 30 repository and combination cells was cheaper, and Pass@1 went up in four of five:

Agent and model	Cost change	Pass@1 change
Claude Code, Sonnet 5	23.6% lower	+1.3
Claude Code, Opus 4.8	30.9% lower	-1.0
Codex, GPT-5.6 Terra	24.7% lower	+0.9
Codex, GPT-5.6 Sol	26.1% lower	+1.6
Copilot CLI, Sonnet 5	18.4% lower	+2.6

The README also reports 16 to 21% lower cost on Web-Bench and 6 to 20% on two security-auditing benchmarks where every task runs in a new repository. These are the authors' numbers on benchmark sequences; yours depend on how often you return to the same repository.

Does It Need Re-Initializing After Every Update?

Partly, and only for one mechanism. For Claude Code and the Copilot CLI, the install steps say to run /tokeneconomizer:toolprune refresh-catalog once per machine and again after the agent updates. The command records which tools a session loads, with no model call. The plugin docs are blunt about skipping it: without the catalog, the deny list sees only the few tools Claude Code defers, not the ones every session loads. A CLI update can add or rename built-in tools, so the old catalog goes stale.

The docs do not say an update resets the learned state. What the plugin learns stays in ~/.tokeneconomizer, kept per agent and per repository.

The bigger update risk is the version range. The README lists Claude Code 2.1.300 to 2.1.310, Codex 0.153.1 to 0.164.0 and Copilot CLI 1.0.83 to 1.0.102 as supported; newer versions are untested. The plugin depends on agent hooks and transcript formats, which are not stable on every host. An open Hermes issue shows what that looks like: on Hermes 0.23.0, newer than the 0.22.5 the plugin was verified on, plugin hooks never reach the adapter on one-shot runs, so no session is learned from. The README promises that if any part of the plugin fails, the session carries on as if it were not installed. That is the safe failure, but it is also a silent one: you keep paying and the plugin stops saving.

Two Failure Modes to Check

The plugin costs more than it saves. In issue #87, a user with 31 MCP servers and 503 tools found that each side call kept every MCP server attached. A one-line Haiku call cost about 572k effective tokens by the plugin's own accounting, against about 14k with the servers removed. The issue was fixed and closed the same day, but the lesson stands: a cost controller that calls a model through your agent inherits your agent's context weight. Watch the plugin overhead line in status.

The plugin sees less than you think. In issue #113, a user reported 15 sessions, 161.5M effective tokens, 917k tokens of plugin overhead (0.6%), and zero facts, pruned packages or redirects. The maintainer replied that the hooks had seen about 14 turns out of 178 prompts, likely because the sessions were resumed from conversations started before the plugin was on, and that a fact must recur across several sessions before it is injected, and each mechanism acts only when its kind of waste shows up. Zero actions is not a bug, but it is not savings either.

Pattern Caching or a Newer Model?

A model upgrade affects each mechanism differently. Tool definitions are sent on every request whatever the model, so pruning keeps paying. Repository facts stay true when the model changes; a smarter model may need fewer of them and loop less, and then those mechanisms simply act less. In the paper the largest saving, 30.9%, came with Opus 4.8, so a high-end model does not remove the need. Use this table to decide:

Your situation	Better first move
Same few repositories every day, long sessions, many MCP servers	Install TokenEconomizer, keep tool pruning on
Agent keeps re-trying the same broken build or test command	Turn on micro-skills (envfacts first, no model calls)
Task success is the problem, not cost	Move to a stronger model; pruning cannot fix wrong answers
Mostly one-off work in new repositories	Expect smaller savings (6 to 20% in the authors' security runs)
CLI auto-updates past the tested range	Pin the agent version or re-check status after each update
A newer model is cheaper per solved task	Switch models and keep the plugin; the savings stack

The last row is the honest answer to "will it survive the next model release". Cost per solved task, not price per token, decides whether a new model is cheaper. Our SRE Bench leaderboard reports cost per root cause next to accuracy for exactly this reason. A plugin that removes repeated waste is a multiplier on whatever model you choose, not a rival to it.

A Checklist for Running It

1. Install, then run status to confirm the hooks fire.
2. On Claude Code and Copilot, run toolprune refresh-catalog now and after every CLI update.
3. Use toolprune=propose if you want to review the deny list before it applies.
4. On Claude Code, turn on history_import right after installing to learn from earlier sessions.
5. Start new sessions rather than resuming pre-install conversations.
6. After a week, read the overhead share and the counts of facts, pruned tools and redirects.
7. Keep the agent inside the supported version range, or re-check after each update.

By default the deny list is written once the repository's sessions have been idle for 60 minutes (or a day after the list formed, at the latest), because a conversation resumed after its tools change must write its whole history to the prompt cache again. For why that matters, see where AI agent tokens go.

The Same Idea in Incident Response

TokenEconomizer bets that repeated sessions on the same system should get cheaper, as long as learned knowledge needs evidence and can be retired. Akmatori applies a similar rule to incident investigations. As the architecture docs describe, a post-incident memory pass saves at most five memories per closed incident, each memory carries an evidence grade, and the Dreaming cron job can retire unverified agent memories nobody used for 90 days. Tools work like pruning: disable a tool instance and it leaves the generated connected-tools list the agent reads.

Conclusion

TokenEconomizer, as its authors describe it, is not a response cache and replays nothing. It learns repository facts, prunes unused tools and redirects stuck turns, and its authors measured 18.4 to 30.9% lower cost on SWE-Bench sequences. After an update, re-run the tool catalog, not the whole setup; the real risk is a CLI version outside the tested range. Measure its overhead in your own repositories, and treat it as a complement to picking the model with the best cost per solved task.

If your team wants AI-assisted incident workflows on your own infrastructure, Akmatori helps SRE teams investigate alerts, coordinate response, and automate safe infrastructure actions. Powered by Gcore for global infrastructure reliability.