Google Open-Sources Harness²: Improve Agents Without Retraining Weights

Google Cloud AI Research has open-sourced Harness² (Harness2), a plug-and-play framework for recursive agent harnessing. The idea is simple and builder-facing: an agent is Model + Harness. You leave the model weights frozen and improve the system around it — prompts, guardrails, memory, tools, skills, scripts, subagents, and plugins — then learn from that improvement process so the next task gets better guidance.

The code is on GitHub under Apache 2.0, with a project site and an arXiv paper. The README states this is not an officially supported Google product. Lead author Ruiyao Xu (Northwestern University) worked with Google Cloud AI Research (Yanfei Chen, Tomas Pfister, and co-authors). Every benchmark figure below is paper- or README-reported. 9to6AI has not hands-on tested Harness².

Key points

• Agent = Model + Harness. Weights stay fixed; the editable surface is the harness.
• Two recursion levels. Task-level Propose–Probe–Compose refines the execution harness for the current task. Harnessing-level recursion updates a persistent improvement library with adopted harnesses and evidence-supported lessons.
• Eight editable components: system prompt, guardrails, memory, tool guidance, subagents, skills, scripts, and plugins.
• Paper-reported gains (single step, k=1): up to 11.5 points on WorkBuddy-Bench (GLM-5 / OpenCode 64.2 → 75.7), up to +4.5 on JobBench, and up to +11.8 pp on LAB Partial. Parallel recursion at k=5 adds further gains (e.g. Gemini 3.5 Flash / OpenCode WorkBuddy average 72.4 → 77.5 → 80.8).
• Executors: OpenCode (default) and Codex. Benchmarks: JobBench, LAB, WorkBuddy-Bench.
• Local demo: Python 3.12 + uv; uv pip install -e '.[render]'; harness2-demo needs no model credentials.

What a harness is — and why builders care

In coding agents and professional workflows, performance often swings more on scaffolding than on swapping the base model. The harness is that scaffolding: how context is built, which tools are used and how, how work is delegated, and what runs inside the tool loop.

Harness² treats harnessing — optimizing that layer — as something you can run recursively. Instead of only hand-tuning a system prompt once, the system proposes edits, probes them on the same task, and composes a refined harness from behavioral contrasts. It does this without ground-truth verifiers for the open-world case the paper targets: tasks that were not seen when the harness was designed.

For teams shipping agents, the practical pitch is: keep your model, spend recursion budget on the harness, and accumulate reusable improvement experience across a stream of tasks.

How Harness² works

Task-level recursion: Propose–Probe–Compose

For each task, the pipeline starts from a common base execution harness and runs a contrastive step:

1. Propose — After a reference run under the current harness, the improver drafts candidates. One targets invariant agentic weaknesses (planning, tool use, context, delegation). Another targets domain-specific knowledge and procedures.
2. Probe — Each candidate runs on the same task under matched settings. Trajectories and deliverables are compared with the reference to see whether the edits helped, did nothing, or regressed.
3. Compose — Supported edits are merged, revised, or discarded based on those contrasts. The composed harness is executed for the final deliverable.

You can scale that step. Parallel recursion explores multiple invariant and domain candidates and compositions at budget k. Sequential recursion carries the refined harness and accumulated evidence through k successive steps.

Harnessing-level recursion: the improvement library

After the task, Harness² does not simply inherit the last execution harness for the next job. Each new task resets the executor to the base harness. What carries forward is the improvement harness: a library of adopted harnesses plus EXPERIENCE.md lessons about when and why changes worked. Later Propose / Probe / Compose steps can reuse or adapt those components.

The eight editable components

Harness² exposes eight file-level components around the model:

Edits are diffs to these files, not weight updates.

Key paper results (vendor / paper reported)

Across three professional benchmarks and four model–harness configurations (GLM-5 and Gemini 3.5 Flash under OpenCode and Codex), the paper reports that a single Harness² step improves over the base harnesses:

Baselines in the paper include ReasoningBank, Meta-Harness (offline), and Harness Scaling. Harness² is framed as online, label-free refinement over a task stream with an improver library that updates after each task. Default experiments use a strong harness editor (Claude Opus 4.8 under OpenCode in the paper’s setup); the paper also reports gains when the same model serves as both executor and editor.

Treat these as research-reported numbers under the authors’ harnesses, judges, and budgets — not as a guarantee for your production agent. Recursion adds inference cost (extra proposals, probes, and editor sessions). Parallel sessions can cut latency when concurrency is available; they do not remove the extra compute.

How to try it

From the README, the local path is:

# Linux, Python 3.12, uv
uv venv --python 3.12 .venv
source .venv/bin/activate
uv pip install -e '.[render]'
harness2-demo --output runs/demo --k 2

The demo runs both recursion modes with deterministic editors and real Python executions and needs no model credentials. Model-backed benchmarks need Node.js 20+, Google Cloud CLI / Vertex setup (or alternatives documented in the repo), and per-benchmark deps (LAB: Pandoc and Poppler; WorkBuddy-Bench: Docker). Executors supported in the release: OpenCode and Codex (--substrate cx).

Repo: github.com/google-research/harness2 · Site: harness2.github.io · Paper: arXiv:2610.14224

What builders should know

1. This is harness engineering, not another fine-tune. If your bottleneck is tool discipline, verification, or domain procedure — not raw model IQ — Harness² is aimed at that layer.
2. Start with the credential-free demo before spending Vertex or API budget on full benchmarks.
3. Budget k carefully. Paper gains grow with parallel recursion at k=5, but each step multiplies rollouts and editor work.
4. Network access is unrestricted in the stock setup. The README warns that public rubrics may be reachable online; configure network restrictions before evaluation so agents cannot fetch answers or grading criteria.
5. Not an official Google product. Apache 2.0 code from Google Research / Cloud AI Research collaborators, with the usual open-source disclaimer and no Google OSS vulnerability-rewards eligibility.

What to watch

• Independent reproductions on WorkBuddy-Bench, JobBench, and LAB outside the paper’s setup.
• Whether improvement libraries transfer cleanly across domains and executors in other people’s stacks.
• Adoption inside OpenCode / Codex workflows versus custom agent runtimes.
• Cost curves: when recursion budget stops paying for itself on real workloads.

Key takeaways

• Google Cloud AI Research collaborators open-sourced Harness² for recursive improvement of agent harnesses with frozen model weights.
• Two levels: task-level Propose–Probe–Compose, and a harnessing-level improvement library across tasks.
• Eight editable components cover context, tools, orchestration, and middleware — not just the system prompt.
• Paper-reported gains reach +11.5 on WorkBuddy-Bench, +4.5 on JobBench, and +11.8 pp on LAB Partial; parallel k=5 adds more on the reported WorkBuddy averages.
• Install with Python 3.12 + uv; local harness2-demo needs no credentials. License: Apache 2.0. Not an officially supported Google product.
• 9to6AI has not hands-on tested the framework; numbers are paper/README claims.

Sources

• Harness² GitHub repository (google-research/harness2)
• Harness² project site
• arXiv:2610.14224 — Harness²: Recursive Agent Harnessing for an Open World
• README (raw)