← Back to blog
AiAbout 6 min read

If Your Agent Follows the Same Script Every Time, Compile the Script

Published Nov 13, 2026
If Your Agent Follows the Same Script Every Time, Compile the Script

Most production agents are a language model reading a procedure and improvising the steps. Someone wrote a document explaining how to handle a refund, or triage a ticket, or reconcile an invoice, and the model reads it at runtime and decides what to do next at every branch.

A paper called ReTrace asks the obvious follow-up question. If the procedure does not change between runs, why is a model deciding the steps every time?

The claim

The reported numbers are specific enough to argue with, which is a good sign. On 79 production procedures, workflows generated by ReTrace completed 93.8% of tasks. The original agents completed 96.8%. The compiled workflows cost 24% less per task.

On a separate benchmark the timing gap is much wider: 1.3 seconds versus 170 seconds for the agent equivalent.

Read those together and the shape of the trade appears. You give up three points of completion and you get a task that costs a quarter less and finishes in about one percent of the wall time. For a workflow running ten thousand times a day, that changes what the workflow is for.

How it compiles

The mechanism is closer to extracting a program than to prompting better.

The system watches an agent follow a procedure and extracts the reasoning behind each tool call. Those reasons are not kept as prose. Conditions that recur across runs get merged into a compact rule document, and that document is compiled into an executable workflow: a graph of steps, branches, and tool invocations that runs without a model in the decision path.

The result is that the model's judgment is captured once, at build time, and reused. At run time the only thing executing is code that was derived from that judgment.

If this sounds like the argument for ahead-of-time compilation over interpretation, that is because it is. An interpreter is flexible and pays for the flexibility on every execution. A compiler pays once and then runs fast, and breaks when the input stops matching what it was compiled against.

Where the three points go

The honest question is which 3% of tasks the compiled workflow drops, and the paper's framing suggests the answer is the unusual ones.

A procedure document covers the normal path and gestures at the exceptions. An agent reading it can improvise when a case does not fit: ask a clarifying question, try a different tool, notice that the customer's account has a flag that changes everything. A compiled workflow does exactly what it was compiled to do, and when a case falls outside the rule document it either fails or does something wrong confidently.

That is why the 24% cost saving is interesting and the 93.8% is the number that decides adoption. In a workflow where a failed task is retried by a human at ten dollars a case, saving 24% of compute while creating 3% more escalations can be a net loss. In a workflow where a failed task is caught by a downstream validator and retried automatically, the same numbers are free money.

The paper does not appear to break the failures down by type, at least not in the coverage available. That breakdown is what a team would need before moving a procedure into production.

What this says about the agent stack

The framing that has dominated the last two years is that agents get better as models get better. ReTrace points at a different axis. Some of the work agents do is genuinely open-ended, and needs a model in the loop on every step. Some of it is a procedure with a fixed decision tree, and running a model on it is a waste of tokens in the same way that calling a language model to parse a date is a waste of tokens.

The interesting part is that the split is discoverable rather than declarative. You do not have to decide in advance which of your workflows is procedural. You run the agent, watch what it does, and let the extraction step tell you whether the decision pattern is stable enough to compile.

That has an organizational consequence. The compiled workflow is a readable artifact. Someone can open it and see the actual rules, including the ones nobody documented, like "if the refund is over five hundred, route to a human" that exists only because the agent inferred it from examples. That is a better audit trail than a transcript, and it is the kind of thing a compliance team can sign off on.

The maintenance problem nobody has solved

Compiled workflows have a failure mode that agents do not: drift. Change the refund policy and the compiled graph keeps executing the old one until someone recompiles it. Change the tool schema and the workflow calls a function that no longer accepts those arguments.

An agent reading a procedure document has the same problem in a milder form. It reads the current document, so a policy change propagates on the next run. It also reads the current tool schema, so a renamed parameter gets handled. You pay for that adaptability on every call, and ReTrace's whole argument is that you are overpaying.

The fix is obvious and unglamorous: recompile on change, and treat the rule document as a versioned artifact under the same review process as code. Whether the tooling makes that easy is the difference between a research result and a product.

Should you try it

If you run anything at volume that follows a written procedure, the calculation is worth doing on your own data. Take one workflow, log a few hundred runs, and look at how often the agent took a path that was not in the document. If the answer is close to never, you are paying a model to be a switch statement.

If the answer is often, your procedure is less of a procedure than you thought, and compiling it will produce a workflow that fails on the cases your business actually cares about.

The paper's most useful contribution may be that it makes this measurable. For two years the argument about agents has been about capability, and capability is hard to price. ReTrace prices a specific decision: how much is a model's judgment per step worth, in dollars and in seconds. On 79 procedures the answer came back at 24% of the compute and 99% of the latency, for three points of completion. That is a number a finance team can work with.

The full paper is on arXiv, and the coverage available so far is thin on implementation details, so treat the figures as a claim rather than a recipe. But the question it asks is the right one, and every team running agents at scale is about to be asked it by someone holding an invoice.

Related articles