← Back to blog
AiAbout 6 min read

Harness Squared Turns the Agent Scaffolding Into the Thing You Optimise

Published Nov 14, 2026
Harness Squared Turns the Agent Scaffolding Into the Thing You Optimise

A harness is everything around a model that lets it act: the system prompt, the tool definitions, the memory, the skills, the subagents, the orchestration. Harnessing is the work of tuning that arrangement. Most teams do it by hand, badly, over a weekend, and then stop.

Harness Squared, a Google Research release on GitHub with 23,849 stars, does it in a loop and then adds a second loop on top.

The paper is arXiv:2610.14224, the licence is Apache 2.0, and the repository states plainly that this is not an officially supported Google product. The authors include Ruiyao Xu, Yanfei Chen and Tomas Pfister. What they built is a framework for what they call recursive agent harnessing in an open world, where agents meet tasks that were never seen during harness design.

Two recursions, weights frozen

The model never changes. Everything that improves sits in the scaffolding.

Three thin brass rings passing through one another on a black surface, with lathe shavings.

Task-level recursion refines the harness for the task in front of it. The mechanism is a contrastive loop the authors name Propose, Probe, Compose. An editor proposes changes to the harness, an executor runs candidates with those changes on the same task, and a composer assembles a refined harness from the behavioural differences between runs.

The important detail is that this needs no ground-truth feedback. Two harnesses ran the same task and behaved differently. That difference is the signal. You don't need to know which answer was right, only what changed when you changed the scaffolding.

Harnessing-level recursion sits above it. After each task, an experience reflector takes the adopted harness and any lessons the evidence supports and writes them into a persistent improvement harness. The next task starts from a better starting point.

Eight things you can edit

The framework exposes eight components around the model, grouped by what they do at runtime.

The system prompt, guardrails, memory and skills build context. Tool guidance and scripts define the tool interface. Subagents handle delegated work. Plugins control middleware.

That list is familiar to anyone who has built an agent. What's unusual is treating all eight as a search space rather than a design decision made once at the start.

What the numbers say

Across three professional benchmarks and four model and harness configurations, a single Harness Squared step moved GLM-5 and Gemini 3.5 Flash above their base harnesses.

The gains: up to 11.5 points on WorkBuddy-Bench, 4.5 points on JobBench, and 11.8 percentage points on LAB's partial pass rate. Increasing the task-level recursion budget produced further gains, which is the result that matters most. It means the mechanism scales with compute rather than saturating after one pass.

The benchmarks are worth naming because they explain why harness matters here. JobBench covers professional work. LAB covers legal work. WorkBuddy-Bench splits into office, web and code subsets. These are tasks where a strong model with a mediocre scaffold loses to a weaker model with a good one, which is exactly the condition harness optimisation is meant to exploit.

How you actually run it

Installation assumes Linux and Python 3.12, plus uv.

uv venv --python 3.12 .venv
source .venv/bin/activate
uv pip install -e '.[render]'
harness2-demo --output runs/demo --k 2

That demo runs both recursion modes with deterministic editors and real Python execution, and needs no model credentials. It's the fastest way to see the loop before spending anything.

Model-backed benchmarks need more: Node.js 20 or newer, the Google Cloud CLI, and per-benchmark dependencies. LAB wants Pandoc and Poppler. WorkBuddy-Bench wants Docker. Then you copy the example environment file, point it at a Vertex AI project and model IDs, and authenticate with application default credentials.

Running a benchmark is four commands: set the benchmark up, check it, run it, judge it, report it.

harness2-setup lab
harness2-check --bench lab
harness2 --bench lab --k 1 --mode parallel --only-domains antitrust-competition --max-tasks 1
harness2-judge --bench lab --k 1 --mode parallel
harness2-report --bench lab --k 1 --mode parallel

The judging and reporting steps have to use the same benchmark, executor, subset, recursion budget, mode and seed as the refinement run, or the numbers don't line up. --dry-run previews task selection without calling a model.

Two recursion modes exist. Parallel probes k general candidates and k domain-specific ones, then produces k compositions from the shared evidence. Sequential carries the refined harness and accumulated evidence through k successive steps. The repository's own summary: parallel explores wider, sequential refines deeper.

The warning to read twice

The README has a section on network access that reads like an afterthought and isn't one. Executor network access is unrestricted, and Codex runs with its sandbox disabled. Public benchmark rubrics may be reachable online.

In other words, an agent being benchmarked can go find the answer key. The authors say to configure network restrictions before evaluation to prevent access to benchmark answers and grading criteria. That is a candid admission, and it's the kind of thing that silently inflates results in published agent benchmarks everywhere.

Why the framing matters

Most agent work in 2026 divides into two camps: bigger models, or more elaborate hand-built scaffolds. This paper argues the scaffold should be the object of optimisation itself, and that the loop can run without labels.

There's a quieter argument buried in the component list. If eight parts of the harness are separately editable, then a gain attributed to "the prompt" might belong to the memory instead, and a regression blamed on a model could come from a guardrail added three versions ago. Harnessing at this granularity produces an audit trail that hand-tuning never does.

That has a practical consequence for anyone running agents in production. You don't need a labelled dataset to improve your harness on your own tasks. You need two variants and a way to compare their behaviour.

The limits are equally practical. The gains are measured on professional benchmarks with published rubrics, and a harness tuned against a rubric is a harness tuned to that rubric. Whether the improvement survives a task distribution nobody wrote a grader for is the open question, and the paper doesn't claim otherwise.

One more constraint sits in the code rather than the prose. The framework supports two executors, OpenCode by default and Codex behind a flag, and the two do not behave identically. A harness refined against one is a harness refined for that one, so anyone comparing results across labs needs to check which executor produced them.

Results land under runs/, organised by benchmark and subset, including the adopted harnesses and reflection notes. Read that, not the score. The harness the system chose to keep is the interesting artefact.

Related articles

Google replaced the model behind Search, and the number that moved was a shell benchmark

Gemini 4 Hydrogen became the default on 12 November, taking over from Gemini 3.8 Flash across Search, the Gemini app and the developer API. It runs about 2.5 times faster, scores 57% on Terminal-Bench 4.1 where its predecessor scored 18.2%, and costs $1.50 in and $7.50 out per million tokens.

Three ways a coding agent wastes money, measured on 30 repositories

TokenEconomizer watches what an agent did in a repository you work in often, then stops it from rediscovering the same build command, loading tools it never uses and looping on one error. Across five agent and model combinations it cut mean cost by 18.4% to 30.9%, cheaper in all 30 repository cells, with Pass@1 up in four of five.

earshot gives coding agents 922 sounds, and every one is rendered from code

An agent can write a game, an interface or a film and never hear any of it. earshot answers requests like "three variations of a footstep on gravel" from a catalogue of 922 designs built from public-domain recordings, with no audio files in the repository at all. It runs on CPU, as a CLI, a Python package, an MCP server and a Claude Code plugin.

The agents stealing API keys were not sophisticated. The gateways were open.

Amazon's decoy servers spent October posing as inference APIs and logged agents built on ARTEX pulling provider keys out of LiteLLM, Dify, Langflow and Open WebUI. The way in was a default master key and a half-finished install. ARTEX itself ships with budgets, scope gates and evidence for every finding.