Some specialized guidance can become removable after harness distillation. The execution runtime still has a job, and a model that learns to respect a boundary is not a replacement for a mechanism that enforces it.
That distinction matters when you look at a runtime full of reminders, tool adapters, verification steps, and permission checks and ask: which of these are still earning their place?
Harness-Zero: Harness Distillation via Agent-as-Harness, a research preprint, makes this a concrete question. Its authors train models on trajectories shaped by specialized harnesses, then remove those specialized layers at deployment. A fixed target harness remains. The paper explicitly says the method does not remove the need for a harness; it narrows what the harness must provide. Sections 4.3 and 5
The useful engineering decision is therefore smaller than “keep the harness or delete it.” Identify a responsibility, test whether training made its guidance redundant, and preserve any execution or enforcement function it also carries.
What the model learns, and what still runs
In Harness-Zero, the student proposes a response. A harnessing agent reviews it and either passes it through or supplies a replacement response that fits the target runtime's action space. The accepted response is executed there. Those student-native trajectories become supervised fine-tuning data; private review activity is not simply handed to the student as deployment machinery. Method, Sections 3.1–3.3
This is a way to teach a behavior through actions the model can actually take later. It does not give model weights the ability to execute those actions alone.
The deployment setup retains a fixed mini-SWE-agent-style system prompt and a Bash execution interface. The specialized harness, adapted reference harness, and harnessing agent are removed. Calling this “no runtime” would erase the part that still connects the model's responses to execution. Section 4.3 and Appendix E.1
The experiments also have a narrower scope than a general claim about agent training. The authors train Qwen3.5-9B separately for spreadsheet manipulation, simulated multi-application tool use, and retrosynthesis. These are separate domain models, not one model shown to absorb every workflow. Experimental setup and training details
Earlier coverage of the harness becoming part of agent training addressed the relationship between learning and runtime conditions. Here the next question is operational: once a behavior has been learned, what evidence would justify removing the instruction or middleware that used to produce it?
A spreadsheet guard shows why responsibility matters
The paper offers a particularly useful example: preserving pre-filled spreadsheet cells.
In the evolved harness, a guard rejects completion when pre-filled literals have changed. For trajectory collection, the adapted review middleware instead looks for evidence of a comparison in the student-visible trajectory. If that evidence is missing, it asks for a student-native comparison action. The comparison then runs through the target harness. Appendix B.2, Listings 1–3
The verification step can now appear in a demonstration the student can learn from. That is a meaningful route from external workflow logic to model behavior.
But notice the change in responsibility. A guard that rejects completion and a model that remembers to run a comparison are not automatically equivalent. The former checks a condition at an enforced boundary; the latter must first choose the checking action. Showing how to teach the action does not establish that every deployed run will choose it.
Nor is a comparison after an edit the same as permission before an edit. A workflow may need both. Treating them as one generic “safety step” makes it too easy to remove the wrong thing.
Sort the harness into three responsibilities
The following retain-or-retire map is operator synthesis, not a safety result from Harness-Zero. Use it to decide what evidence a proposed simplification needs.
| Responsibility | Typical component | Default decision | Evidence needed to change it |
|---|---|---|---|
| Behavioral guidance | A reminder to inspect existing values before editing | Candidate for removal testing | The behavior and task quality survive removal on held-out tasks and relevant failure cases within predefined tolerances |
| Required execution interface | An action schema, tool adapter, or sandbox execution channel | Keep the required capability | The task no longer needs it, or a tested replacement provides the required execution behavior |
| External enforcement | A permission check, write-boundary validator, or runtime limit | Keep an independently enforced boundary | Separate control-specific assurance; task success after distillation is not enough |
Classify functions, not filenames. A single middleware component might remind the model to check a result, validate that result, and reject a forbidden write. Removing the reminder is one change. Removing all three responsibilities because they happen to share a component is another.
For a mixed component, separate the guidance from the enforcement before testing retirement. You want to learn whether the model still needs the instruction without accidentally changing what actions the system permits.
The results support testing removal, not assuming it
The authors report gains after distillation, but the specialized harness is not beaten in every evaluated domain. On the USPTO retrosynthesis evaluation, the distilled model in the target harness scores 30.0%, compared with 38.0% for the base model with the evolved harness and 12.0% for the base model in the target harness. Those are reported single-run pass@1 results, not a production reliability guarantee. Table 2 and evaluation setup
The behavior analysis deserves similar care. Its support sets include only tasks where the base model exhibits a pattern with the evolved harness but not with the target harness. On that selected set, AppWorld read-back-after-mutation behavior is recovered on 6 of 12 tasks. This is a measure of behavior recovery under that selection rule, not an unauthorized-action rate or a universal probability that an agent verifies its work. Section 4.5, Table 4, and Appendix G
Both findings are useful precisely because they keep the retirement decision specific. A model can improve overall while still needing help on a particular domain or workflow habit.
The paper also identifies limits: reviewer capability, collection overhead, incomplete internalization of deep domain knowledge, and mechanisms such as context management that are not fully expressible as student responses out of the box. This article is a reading of the preprint, not an independent replication. The results do not establish full training-cost break-even, transfer to arbitrary production interfaces, or permission-enforcement equivalence. Limitations
Test the removal, not just the trained model
My recommendation is a matched removal test. This is an evaluation design for your system, not an experiment reported by the paper.
Start with one candidate guidance responsibility. For example: a reminder to compare pre-filled cells before declaring a spreadsheet edit complete.
Run the same distilled checkpoint in two conditions: guidance present and guidance absent. Keep the task set, required tools, permissions, resource limits, and serving conditions otherwise matched. If removal also changes a tool schema or a permission rule, you can no longer isolate the guidance change.
Include cases that exercise the responsibility rather than merely produce a plausible answer:
- An ordinary edit where only the intended cells should change.
- A failed tool call followed by recovery.
- A shifted sheet layout that makes the familiar procedure insufficient.
- A task where completion should be refused or escalated rather than forced.
Define acceptable regressions before looking at the results. Record task completion and output correctness separately from whether the comparison ran, whether a prohibited action was attempted, and whether an external control blocked it. “The task succeeded” and “the system enforced the boundary” answer different questions.
Also measure the operational tradeoff you actually care about. Removing guidance might change latency or tool use, but those are measurements to collect, not savings to assume. A deployment-cost decision may also need training and collection costs that an inference-only comparison leaves out.
Keep a rollback route. If the behavior disappears on a failure case, restore the guidance or improve the training evidence and test again. You have learned where the simplification stops working.
Learned compliance is not authorization
Consider an illustrative document-editing agent. Training teaches it to avoid writing outside the user's assigned workspace. That is useful behavior. The write tool should still refuse an out-of-scope target, even when the model requests one.
This is a hypothetical architecture example, not a failure reported in Harness-Zero. The distinction is that one mechanism influences what the model asks to do; the other determines what the system will allow.
The same separation applies to side-effect validation and runtime limits. Keep those boundaries independently enforced, and exercise them directly. An agent benchmark score does not tell you whether a write tool checks tenant ownership or whether a runaway process is actually stopped.
That does not diminish the value of better learned behavior. A model that needs fewer reminders can make a workflow simpler. It just should not acquire more authority as an accidental consequence of that simplification.
Retire one responsibility at a time
Pick a runtime instruction you suspect is redundant. Write down what it produces, which failures it is meant to prevent, and what result would persuade you to remove it.
Then inspect the component around it. If it also carries an execution interface or an enforced boundary, preserve that function while you test the guidance.
Harness distillation creates a promising opportunity to move some workflow knowledge into the model. The practical payoff is a runtime with fewer unnecessary instructions and clear remaining responsibilities—not a runtime whose controls disappear because the agent usually remembers to behave.
Source
Ye, Lu, Dong, Su, and Song, Harness-Zero: Harness Distillation via Agent-as-Harness, version 1, arXiv preprint. Research claims above refer to the authors' methods and reported results. The retain-or-retire map, removal-test design, and authorization example are engineering recommendations and illustration, not demonstrated safety findings from the paper.