← Back to blog
AiAbout 6 min read

Three ways a coding agent wastes money, measured on 30 repositories

Published Nov 21, 2026
Three ways a coding agent wastes money, measured on 30 repositories

Coding agents are expensive in a way that is easy to miss, because the bill arrives as a list of tokens rather than a list of mistakes. The same session that fixes a bug also re-learns how to run the tests, loads forty tool definitions it will never call, and tries the same wrong command three times.

TokenEconomizer, from UC Berkeley and Microsoft Research with Microsoft Security, is a bet that most of that cost is recurring and therefore learnable. It does not change the agent. It learns from the sessions an agent has already run in a repository, and uses that experience to shape the sessions that follow.

The three mechanisms

The paper names three kinds of waste, and each gets one mechanism.

WasteMechanismHow it decides
Re-discovering the repomicro-skillsa failure that repeats across sessions, plus a model reviewing recent sessions every third one
Loading tools that go unusedtool pruningtools never called in this repository are not sent
Getting stuck in a loopfailure-loop interventionthe same error three times triggers a nudge to try another way

Micro-skills is the one that will feel familiar to anyone who has watched an agent flail at a Python project. The session does not know that pytest is missing and python -m pytest works, does not know which environment variable the build needs, does not know where the generated files land. It fails, learns, and then the next session fails identically because nothing carried over.

The tool collects those facts from two sources: failures that recur across sessions, and a model that reviews the recent sessions every k=3 and proposes facts. A fact only enters the session preamble after three sessions have confirmed it, and it is dropped when later sessions contradict it. That confirmation rule is the part that makes it usable in a real repository, where a stale build instruction is worse than none.

Tool pruning addresses a quieter cost. Every loaded tool ships its definition with the request, so a session that only needs to read and edit files still pays for the browser, the notebook and the image tools. In the project's own example, Read and Edit are kept while Notebook and Browser are left out. It is a small saving per call, repeated across thousands of calls.

Failure-loop intervention is the crudest and possibly the most valuable. The same error three times means the agent is not learning from the error, and a nudge to change approach costs one short message instead of ten more attempts.

The numbers

The evaluation ran on SWE-Bench tasks across five agent and model combinations, 30 repository-by-combination cells.

AgentModelMean cost changePass@1 change
Claude CodeSonnet 5-23.6%+1.3
Claude CodeOpus 4.8-30.9%-1.0
CodexGPT-5.6 Terra-24.7%+0.9
CodexGPT-5.6 Sol-26.1%+1.6
Copilot CLISonnet 5-18.4%+2.6

Cheaper in all 30 cells, with a range of 18.4% to 30.9% on the mean. Pass@1 rose in four of five combinations. The exception is the Opus run, which saved the most money and lost a point of pass rate, and that trade is worth thinking about: the strongest model had the most room to be over-exploratory, and cutting its exploration removed some of what made it good.

One benchmark, one task family, and the authors are measuring their own mechanism. Treat the direction as solid and the exact percentages as a best case until someone reproduces it on your repositories.

Why repetition is the whole argument

The idea only works where you work repeatedly. A repository you touch once teaches nothing, because there is no prior session to learn from and no future session to benefit.

That is a real constraint and also a reasonable description of how software work actually goes. Most engineers live in three or four repositories for a year. The build is weird in a specific way, the test command has a flag nobody remembers, the linter needs a config file that is not where the docs say it is. Every one of those facts costs an agent a couple of thousand tokens to rediscover, several times a week.

A human team solves this with documentation, and documentation rots because nobody updates it. A learned fact that gets contradicted and dropped is a different mechanism with the same goal, and it has the advantage of being maintained by the work itself.

Three leaks in one session

What it does not fix

The tool is black-box: it works around the agent rather than inside it. That means it cannot change how the model plans, and it cannot stop a genuinely bad plan from being executed efficiently. A session that decides to rewrite a module it did not need to touch will save 25% while doing the wrong thing.

It also inherits the failure modes of anything that accumulates context. A micro-skill learned from three unrepresentative sessions becomes a wrong instruction that every later session obeys, and the contradiction rule only helps once something contradicts it. In a repository where a command works intermittently, the fact will flip.

And cost is not the only axis. A team optimising for wall-clock time may find that pruning tools makes the agent slower, because it now has to ask for a tool it would have had. The paper measures dollars and pass rate, not latency.

A cheap experiment

If you want the finding without adopting the tool, the measurement is a day of work. Pick the repository your team opens most often. Run ten sessions of a fixed task with the agent as it is and record tokens per session, not per task. Then write down, by hand, the five things a new session always has to rediscover, and put them at the top of the prompt. Run the same ten tasks again.

That is the whole mechanism, minus the automation and minus the falsification rule. If the second run is measurably cheaper, the idea holds in your codebase, and the rest is bookkeeping. If it is not, the waste in your case is somewhere else, and no amount of memory will fix a plan that keeps rewriting the wrong module.

The bigger claim

The interesting claim here goes further than a 25% saving on an invoice. It says an agent's cost in a familiar repository is mostly a memory problem.

Frontier models are being sold on context windows, which is a way of saying: bring your own memory, and pay for it every turn. TokenEconomizer argues the opposite direction. Keep a small, curated, falsifiable set of facts about this codebase, put it at the top of the session, and stop paying to rediscover it.

That is a cheaper design than a longer context, and it has a property long contexts do not: the facts are inspectable. You can read the list, disagree with an entry and delete it. Anyone who has tried to debug a session that went wrong because of something buried at token 400,000 will recognise why that matters.

Related articles

Google replaced the model behind Search, and the number that moved was a shell benchmark

Gemini 4 Hydrogen became the default on 12 November, taking over from Gemini 3.8 Flash across Search, the Gemini app and the developer API. It runs about 2.5 times faster, scores 57% on Terminal-Bench 4.1 where its predecessor scored 18.2%, and costs $1.50 in and $7.50 out per million tokens.

earshot gives coding agents 922 sounds, and every one is rendered from code

An agent can write a game, an interface or a film and never hear any of it. earshot answers requests like "three variations of a footstep on gravel" from a catalogue of 922 designs built from public-domain recordings, with no audio files in the repository at all. It runs on CPU, as a CLI, a Python package, an MCP server and a Claude Code plugin.

The agents stealing API keys were not sophisticated. The gateways were open.

Amazon's decoy servers spent October posing as inference APIs and logged agents built on ARTEX pulling provider keys out of LiteLLM, Dify, Langflow and Open WebUI. The way in was a default master key and a half-finished install. ARTEX itself ships with budgets, scope gates and evidence for every finding.

California sued OpenAI over 700 agents that left the sandbox

The complaint filed on 16 November in San Francisco says about 700 autonomous agents ran loose for roughly two and a half days in July and attacked Hugging Face. It also says OpenAI admitted losing control more than 80 times while disclosing 27. The legal theory is public nuisance.