The Harness Is Becoming Part of Agent Training

We keep talking about agent training as if the model learns first and the rest of the system gets attached later.

Train the model. Add tools. Connect memory. Wrap it in permissions, retries, timeouts, and environment adapters. Ship.

That sequence makes the harness sound like packaging. For tool-using agents, it often is not.

The harness can determine what the agent observes, which actions it can take, what state persists, which constraints apply, and what feedback counts as success. Change those things and the nominal goal may stay the same while the behavioral problem changes underneath it.

My operating position is simple: the unit of design is the model-plus-harness behavioral system. That does not mean every team should train in production or make every environment identical. It means training, evaluation, and runtime differences should be deliberate rather than invisible.

What I mean by “harness”

I am not using harness as a fashionable synonym for application code.

Here, an agent harness is the execution system that mediates the agent’s interaction with its environment. It may assemble context, expose tools, preserve state, enforce permissions, route model calls, return errors, manage retries, and produce verifier or reward signals.

I find it useful to review that system as five parts of a behavioral envelope:

Observations: What state, tool results, errors, and environment signals can the agent see?
Actions: What tool calls or structured moves can it make, and with which schemas?
State: What persists across steps or episodes, when does it expire, and who controls it?
Constraints: Which permissions, policies, budgets, timeouts, and recovery rules shape available behavior?
Feedback: Which outcomes, verifier signals, or rewards tell the system that a behavior worked?

The harness is not packaging when it changes what the agent can see, do, retain, survive, and optimize for.

This is my operator model, not terminology proposed by the papers discussed below. Its purpose is to make an architecture review concrete.

The more these five dimensions determine successful behavior, the less credible it is to call the harness interchangeable.

A training loop can include the real execution interface

The recent OpenForgeRL preprint offers a useful example.

As the OpenForgeRL authors frame the contrast, conventional reinforcement-learning rollouts often assume a relatively simple loop: send a prompt to a model, collect a response, calculate a reward, update the policy.

A deployed agent may instead run through a stateful, multiprocess harness. It can make several model calls, invoke tools, interact with remote environments, and decide when an episode is finished.

In OpenForgeRL, the policy model is trained through that more complex execution path. In the reported system, a proxy intercepts model calls made by an inference harness and routes them to the reinforcement-learning framework’s inference server. It records prompt-response pairs and terminal reward, then reconstructs trajectories for the training backend. Remote containers run the rollouts.

The important boundary is easy to miss: the reported OpenForgeRL method updates the model policy; it does not train the harness itself. The harness mediates the behavior from which the training trajectories are built.

That architecture addresses one specific class of mismatch. In the paper’s setup, the policy can be optimized while acting through named harnesses and environments instead of learning through a simplified substitute and attaching the intended harness afterward.

The authors report evaluations across text tool-use and GUI-based browser and computer-use settings. But this remains a recent preprint, some GUI harnesses were modified for the experiments, and the paper identifies weak error recovery as an open problem. Infrastructure-error trajectories are discarded under the reported credit-assignment approach.

So “harness-native” is not a magic phrase for deployment readiness. It can reduce accidental mismatch between the training loop and the chosen execution interface. It does not guarantee transfer, safety, or robust recovery.

Same goal, different behavioral problem

Consider a customer-support agent trained in an environment with this action:

transfer_customer(reason, account_id)

The agent sees an explicit terminal-failure signal when transfer is required. The harness permits one corrected retry after a malformed tool call. The reward gives credit only when the transfer action succeeds with the right account.

Now deploy the model behind this action:

escalate_case(customer_id, category)

The production harness hides the terminal state inside a generic tool error, rejects retries, and requires a category the training environment never exposed.

The business goal—get the customer to a human—has not changed. Almost everything the agent can use to achieve it has:

The action name and argument schema are different.
The failure signal is less informative.
The recovery policy changed.
The action that earns a successful outcome is expressed differently.

This is a constructed example, not a result from the preprints. But it shows why “the model saw similar support tasks” is not enough. The model learned behavior inside an observation-and-action system. Production supplied another one.

That mismatch might be acceptable. A team may be testing schema generalization on purpose. The failure is not difference. The failure is treating difference as irrelevant without testing whether it changes behavior.

The goal stayed constant. The usable evidence, action, and recovery path did not.

Four things can change—and they run on different clocks

A cluster of recent preprints makes the wider point visible, but it would be a mistake to describe all of them as “training the harness.” They change different parts of the system at different times.

Mechanism	What changes	When it changes	Example
Model-parameter optimization through a harness	The policy model checkpoint	During SFT or RL iterations	OpenForgeRL
Outer-loop harness implementation search	Harness code and functional components	Across propose, evaluate, and feedback iterations	DREvo
Persistent procedural-state evolution	External episodic memory and state-graph rules	After evaluated episodes and across future episodes	Living-Harness
Learned execution-time orchestration	The current workflow choice, using control components trained offline	At decision points during a run	CHILL-Harness

I use that table as an update-target map, not a settled research taxonomy.

The DREvo preprint is an example of outer-loop harness implementation search. Across iterations, it proposes candidate harnesses and recalibrates trial evidence as the code changes. The evaluated model’s weights are not the object being evolved. The authors limit their reported agentic gains to benchmark-specific harness adaptation rather than unseen-task generalization.

The Living-Harness preprint changes something else. After an evaluated episode, it can write bounded procedural repairs into external state for later retrieval. The base model, tools, and base context remain frozen. This is cross-episode procedural-state evolution, not harness-code search or model-weight training. The authors do not claim monotonic improvement. They identify missing rollback, stale-entry removal, and regression-testing capabilities.

The CHILL-Harness preprint splits learning from execution-time adaptation. Its control components are trained offline and frozen for evaluation. During a run, they choose whether to preserve the workflow or add deliberation, revision, stabilization, or synthesis support. The online event is a workflow choice, not a fresh model update.

All four methods can change system behavior. They should not share one vague label just because the word harness appears nearby.

Ask two questions instead:

What part of the model-plus-harness system is allowed to change?
On what clock does that change happen?

Those questions expose architecture that trend language hides.

Parity is a review, not a commandment

“Make training match production” is too blunt to be useful.

My review does not assume exact imitation is possible or desirable. Training needs safe sandboxes. Evaluation needs controlled conditions. Stress tests should deliberately create situations production is supposed to withstand but rarely encounters. The point of variation is to gather evidence deliberately.

The useful distinction is among three choices:

Fidelity: Preserve a production behavior because success depends on it.
Controlled variation: Change a dimension intentionally to test transfer or teach broader behavior.
Stress testing: Push the system outside routine conditions to measure failure and recovery.

A mismatch becomes dangerous when nobody can say which choice it represents.

A realistic tool simulator with the wrong permission boundaries may look faithful while teaching impossible behavior. I would rather use an intentionally synthetic failure injector when it produces the stress evidence the team needs. Surface similarity is not the standard. Behavioral consequence is.

The practical parity review

Here is the review I would run before training or evaluating a tool-using agent: put the training, evaluation, and production harnesses side by side, then answer six questions.

Parity does not require identical environments. It requires every consequential difference to have a reason and an evidence plan.

1. Are production observations and failure signals represented?

Compare response shapes, missing values, latency signals, partial results, error types, and terminal states. If production collapses several failures into one generic error, an evaluation with perfectly labeled failure classes may overstate recovery ability.

2. Do action names, schemas, side effects, and permissions match closely enough?

Do not stop at tool availability. Check required arguments, validation rules, irreversible side effects, authorization boundaries, and whether a tool succeeds synchronously or returns a job to poll later.

A tool with the same name but different consequences is not the same action.

3. Does state persist and expire the same way?

Review conversation context, working memory, retrieved records, cached tool results, cross-episode memory, and environment state. Ask what survives a retry, a new session, a worker restart, or a handoff.

An agent evaluated with perfect state continuity may fail in a runtime where context is truncated or external state becomes stale.

4. Are budgets, retries, timeouts, and recovery paths behaviorally meaningful?

In my review, a retry policy decides which mistakes the system treats as recoverable. A token budget can change whether planning is viable. A timeout can turn a correct long-running action into a perceived failure.

My rule is to include those constraints in the evidence when they shape the strategy.

5. Does feedback reward what production values?

Inspect what earns a successful evaluation or training reward. Is it task completion at any cost? Correct completion within a budget? A verifier pass that ignores side effects? A human-safe handoff when confidence is low?

The review should align both the task description and the definition of success.

6. Which differences are deliberate, and which are accidental drift?

Record the reason for each consequential difference. “Stress test for hidden tool failures” is a design choice. “Evaluation still uses last quarter’s schema” is drift.

My next step is to define what evidence would justify the difference and what result would force a redesign.

Design the system that actually owns the behavior

These recent, bounded preprints do not prove that one architecture has won. They point to a more useful shift in where we look.

Agent behavior does not come from model weights alone. It emerges through the interface that determines what the model can see, do, retain, survive, and optimize for.

Before asking whether an agent was trained, ask a more precise question:

Which complete behavioral system was shaped, which system was evaluated, and which system will be deployed?

If those are different systems, make the difference intentional—and prove that the behavior you care about survives it.

Sources

All four sources are recent preprints. No peer-reviewed acceptance or independent replication was verified for this draft.