m@berryhill: ~/berryhill.dev/posts/what-can-you-remove-after-harness-distillation.md
~ homeposts/about.md
m@berryhill in ~/posts$ cat what-can-you-remove-after-harness-distillation.md
---
title:  What Can You Remove After Harness Distillation?
date:   2026-09-22
topic:  agentic
read:   11 min
words:  2,449
slug:   what-can-you-remove-after-harness-distillation
views:  live post
tags:   [agentic, llm, evaluation, infrastructure]
---
essay · long read

What Can You Remove After Harness Distillation?

Harness distillation can make some runtime guidance removable. Use a retain-or-retire map to separate learned habits, tool interfaces, and enforced controls.

table of contents
  1. What the model learns, and what still runs
  2. A spreadsheet guard shows why responsibility matters
  3. Sort the harness into three responsibilities
  4. The results support testing removal, not assuming it
  5. Test the removal, not just the trained model
  6. Learned compliance is not authorization
  7. Retire one responsibility at a time
  8. Source

Some specialized guidance can become removable after harness distillation. The execution runtime still has a job, and a model that learns to respect a boundary is not a replacement for a mechanism that enforces it.

That distinction matters when you look at a runtime full of reminders, tool adapters, verification steps, and permission checks and ask: which of these are still earning their place?

Harness-Zero: Harness Distillation via Agent-as-Harness, a research preprint, makes this a concrete question. Its authors train models on trajectories shaped by specialized harnesses, then remove those specialized layers at deployment. A fixed target harness remains. The paper explicitly says the method does not remove the need for a harness; it narrows what the harness must provide. Sections 4.3 and 5

The useful engineering decision is therefore smaller than “keep the harness or delete it.” Identify a responsibility, test whether training made its guidance redundant, and preserve any execution or enforcement function it also carries.

What the model learns, and what still runs

In Harness-Zero, the student proposes a response. A harnessing agent reviews it and either passes it through or supplies a replacement response that fits the target runtime's action space. The accepted response is executed there. Those student-native trajectories become supervised fine-tuning data; private review activity is not simply handed to the student as deployment machinery. Method, Sections 3.1–3.3

This is a way to teach a behavior through actions the model can actually take later. It does not give model weights the ability to execute those actions alone.

The deployment setup retains a fixed mini-SWE-agent-style system prompt and a Bash execution interface. The specialized harness, adapted reference harness, and harnessing agent are removed. Calling this “no runtime” would erase the part that still connects the model's responses to execution. Section 4.3 and Appendix E.1

The experiments also have a narrower scope than a general claim about agent training. The authors train Qwen3.5-9B separately for spreadsheet manipulation, simulated multi-application tool use, and retrosynthesis. These are separate domain models, not one model shown to absorb every workflow. Experimental setup and training details

Earlier coverage of the harness becoming part of agent training addressed the relationship between learning and runtime conditions. Here the next question is operational: once a behavior has been learned, what evidence would justify removing the instruction or middleware that used to produce it?

A spreadsheet guard shows why responsibility matters

The paper offers a particularly useful example: preserving pre-filled spreadsheet cells.

In the evolved harness, a guard rejects completion when pre-filled literals have changed. For trajectory collection, the adapted review middleware instead looks for evidence of a comparison in the student-visible trajectory. If that evidence is missing, it asks for a student-native comparison action. The comparison then runs through the target harness. Appendix B.2, Listings 1–3

The verification step can now appear in a demonstration the student can learn from. That is a meaningful route from external workflow logic to model behavior.

But notice the change in responsibility. A guard that rejects completion and a model that remembers to run a comparison are not automatically equivalent. The former checks a condition at an enforced boundary; the latter must first choose the checking action. Showing how to teach the action does not establish that every deployed run will choose it.

Nor is a comparison after an edit the same as permission before an edit. A workflow may need both. Treating them as one generic “safety step” makes it too easy to remove the wrong thing.

Sort the harness into three responsibilities

The following retain-or-retire map is operator synthesis, not a safety result from Harness-Zero. Use it to decide what evidence a proposed simplification needs.

Responsibility Typical component Default decision Evidence needed to change it
Behavioral guidance A reminder to inspect existing values before editing Candidate for removal testing The behavior and task quality survive removal on held-out tasks and relevant failure cases within predefined tolerances
Required execution interface An action schema, tool adapter, or sandbox execution channel Keep the required capability The task no longer needs it, or a tested replacement provides the required execution behavior
External enforcement A permission check, write-boundary validator, or runtime limit Keep an independently enforced boundary Separate control-specific assurance; task success after distillation is not enough

Classify functions, not filenames. A single middleware component might remind the model to check a result, validate that result, and reject a forbidden write. Removing the reminder is one change. Removing all three responsibilities because they happen to share a component is another.

For a mixed component, separate the guidance from the enforcement before testing retirement. You want to learn whether the model still needs the instruction without accidentally changing what actions the system permits.

Retire a responsibility, not a fileThree responsibility rows mark guidance as test for removal, execution as keep capability, and enforcement as keep the boundary.What can you remove?Operator responsibility map01 GuidanceTest for removalCheck the habit on held-out tasks.02 ExecutionKeep required capabilityWeights cannot execute tools.03 EnforcementKeep the boundaryTask success is not assurance.Mixed component? Split its roles.
A mixed component must keep its execution and enforcement roles even when its reminder becomes redundant.

The results support testing removal, not assuming it

The authors report gains after distillation, but the specialized harness is not beaten in every evaluated domain. On the USPTO retrosynthesis evaluation, the distilled model in the target harness scores 30.0%, compared with 38.0% for the base model with the evolved harness and 12.0% for the base model in the target harness. Those are reported single-run pass@1 results, not a production reliability guarantee. Table 2 and evaluation setup

The behavior analysis deserves similar care. Its support sets include only tasks where the base model exhibits a pattern with the evolved harness but not with the target harness. On that selected set, AppWorld read-back-after-mutation behavior is recovered on 6 of 12 tasks. This is a measure of behavior recovery under that selection rule, not an unauthorized-action rate or a universal probability that an agent verifies its work. Section 4.5, Table 4, and Appendix G

Both findings are useful precisely because they keep the retirement decision specific. A model can improve overall while still needing help on a particular domain or workflow habit.

The paper also identifies limits: reviewer capability, collection overhead, incomplete internalization of deep domain knowledge, and mechanisms such as context management that are not fully expressible as student responses out of the box. This article is a reading of the preprint, not an independent replication. The results do not establish full training-cost break-even, transfer to arbitrary production interfaces, or permission-enforcement equivalence. Limitations

Test the removal, not just the trained model

My recommendation is a matched removal test. This is an evaluation design for your system, not an experiment reported by the paper.

Start with one candidate guidance responsibility. For example: a reminder to compare pre-filled cells before declaring a spreadsheet edit complete.

Run the same distilled checkpoint in two conditions: guidance present and guidance absent. Keep the task set, required tools, permissions, resource limits, and serving conditions otherwise matched. If removal also changes a tool schema or a permission rule, you can no longer isolate the guidance change.

Include cases that exercise the responsibility rather than merely produce a plausible answer:

  • An ordinary edit where only the intended cells should change.
  • A failed tool call followed by recovery.
  • A shifted sheet layout that makes the familiar procedure insufficient.
  • A task where completion should be refused or escalated rather than forced.

Define acceptable regressions before looking at the results. Record task completion and output correctness separately from whether the comparison ran, whether a prohibited action was attempted, and whether an external control blocked it. “The task succeeded” and “the system enforced the boundary” answer different questions.

Also measure the operational tradeoff you actually care about. Removing guidance might change latency or tool use, but those are measurements to collect, not savings to assume. A deployment-cost decision may also need training and collection costs that an inference-only comparison leaves out.

Keep a rollback route. If the behavior disappears on a failure case, restore the guidance or improve the training evidence and test again. You have learned where the simplification stops working.

Change only the guidanceThe same model and task set split into guidance present and absent conditions, then join at a comparison of quality, behavior, and control evidence.Test the removalRecommended evaluation designSame checkpoint + tasksTools and permissions unchangedGuidancepresentGuidanceabsentCompare separate evidenceQuality: correct result?Behavior: did the check run?Control: was a violation blocked?Set tolerances before testing.Regression? Restore guidance.
A removal test isolates guidance; changing permissions at the same time tests a different system.

Learned compliance is not authorization

Consider an illustrative document-editing agent. Training teaches it to avoid writing outside the user's assigned workspace. That is useful behavior. The write tool should still refuse an out-of-scope target, even when the model requests one.

This is a hypothetical architecture example, not a failure reported in Harness-Zero. The distinction is that one mechanism influences what the model asks to do; the other determines what the system will allow.

The same separation applies to side-effect validation and runtime limits. Keep those boundaries independently enforced, and exercise them directly. An agent benchmark score does not tell you whether a write tool checks tenant ownership or whether a runaway process is actually stopped.

That does not diminish the value of better learned behavior. A model that needs fewer reminders can make a workflow simpler. It just should not acquire more authority as an accidental consequence of that simplification.

The tool still says noAn illustrative out-of-scope write request reaches a permission check, is denied, and produces no write.Learned habit ≠ authorityIllustrative forbidden-write pathAgent requests a writeTarget is outside its workspace.requestExternal permission checkIs this target in allowed scope?No → denydenialNo write executedReturn a refusal to the agent.The check runs even whenthe learned habit fails.
Learning changes what the agent asks for; authorization decides which side effects can happen.

Retire one responsibility at a time

Pick a runtime instruction you suspect is redundant. Write down what it produces, which failures it is meant to prevent, and what result would persuade you to remove it.

Then inspect the component around it. If it also carries an execution interface or an enforced boundary, preserve that function while you test the guidance.

Harness distillation creates a promising opportunity to move some workflow knowledge into the model. The practical payoff is a runtime with fewer unnecessary instructions and clear remaining responsibilities—not a runtime whose controls disappear because the agent usually remembers to behave.

Source

Ye, Lu, Dong, Su, and Song, Harness-Zero: Harness Distillation via Agent-as-Harness, version 1, arXiv preprint. Research claims above refer to the authors' methods and reported results. The retain-or-retire map, removal-test design, and authorization example are engineering recommendations and illustration, not demonstrated safety findings from the paper.

m@berryhill in ~/posts$
$ cd ../ · back to posts/