A long-running agent finishes with the wrong answer. The final step contains a confident but false conclusion, so the review starts there.
That is usually the easiest place to inspect. It may also be the least useful.
The last wrong step tells you where the failure became visible. It does not necessarily tell you where the failure became causally active. By the time an agent writes the bad answer, an earlier mistake may already have changed its working state, contaminated an artifact, redirected later tool calls, or survived an apparent correction.
In other words, the wrong final answer may only be the last visible symptom. AI agent debugging has to follow the state and artifacts that produced it.
A terminal score compresses that history into one label: failed. Operators need the history back.
The better debugging object is an error lifecycle: where a consequential error began, which later decisions inherited it, whether the system truly cleared it, and how it reached the terminal result.
Local errors are not automatically critical errors
Long agent trajectories can contain several wrong turns. They do not all deserve the same diagnosis.
One error may be caught before anything depends on it. Another may waste tokens but leave the result intact. A third may be compensated for by a later tool response. A fourth may alter the state every downstream step consumes.
The useful question is not, “Where did the agent first do something wrong?” It is:
What is the earliest error that remains necessary to the best-supported explanation of the final failure?
For this review, a critical error is the earliest unresolved error that remains necessary to the best-supported explanation of the terminal failure.
That wording matters. It does not assume every failed run has one uniquely knowable root cause. The retained evidence may support several plausible causal paths. Important context may be missing. A reviewer should preserve those rival explanations instead of turning the first plausible story into certainty.
In the paper TRAJDEBUG: Tracing Error Lifecycle to Identify Critical Failures in Long-Horizon Agent Trajectories, the authors formalize a related target as the earliest decisive error. TRAJDEBUG is an August 6, 2026 arXiv v1 preprint (arXiv:2608.06346v1) by Yunjia Qi, Zehua Yin, Xintong Shi, Hao Peng, Songyuanyi Lu, Yixian Liu, Richeng Xuan, Yuhong Liu, Zhichao Hu, Xiaozhi Wang, Lei Hou, Bin Xu, and Juanzi Li. Under their counterfactual definition, replacing that step with a correct action—while holding the prior trajectory fixed and allowing the remainder to unfold correctly—would change the run from failure to success.
That is a stronger target than proximity to the final answer. The paper also argues that evidence can be scattered across distant parts of a trajectory and that later errors may be symptoms of an earlier unresolved mistake rather than independent causes.
A failed inventory run has a causal history
Consider a hypothetical inventory-reconciliation agent. Its job is to verify 80 records, update a ledger, and produce an exception report.
Early in the run, it writes an output file under the wrong name. It catches the mismatch before another step reads the file, renames it, and verifies the corrected path. That was a local error, but it did not remain active.
Later, a paginated tool response returns 72 records and a continuation token. The agent treats the first page as the complete inventory and commits a 72-record snapshot to working state.
Everything after that looks locally reasonable:
- calculations use the 72-record snapshot;
- the ledger update covers those 72 records;
- the exception list is generated from that incomplete set;
- the final report summarizes the artifacts it was given.
A later check even notices that the source describes 80 records. The agent explains the eight-record difference as invalid entries, but it never fetches the next page or rebuilds the dependent artifacts.
It acknowledged the contradiction. It did not resolve the error.
The final report confidently says all eligible records were reconciled while eight records are absent. That sentence is the visible symptom. Editing the sentence would not repair the ledger, the exception list, or the incomplete source snapshot they inherited.
If the continuation token, source count, and artifact lineage were retained, the pagination mistake is a strong causal explanation. If they were not retained, it is only the best-supported hypothesis. The reviewer should say what the evidence permits—and what it does not.
What TRAJDEBUG adds to the evaluation conversation
TRAJDEBUG is useful here because it treats failed trajectories as causal histories rather than flat transcripts.
The authors propose a three-stage research method:
- detect error triggers using cited evidence from the trajectory;
- cluster related error instances and classify whether they resolved or remained active;
- attribute the terminal failure from a bounded candidate set.
Multiple levels of history compression keep nearby evidence detailed while summarizing distant history. The paper distinguishes clean resolution, costly resolution, errors that are visibly active, and errors that remain active without obvious symptoms. Those are the paper’s method and labels.
The paper reports exact critical-step identification across 869 failed trajectories assembled from existing benchmarks and a new TrajErrBench set. TrajErrBench contains 486 manually annotated failures: 400 from Tau2Bench and 86 from SWE-Bench Pro. In those experiments, TRAJDEBUG reaches 34.11% macro-average exact-step accuracy, compared with 25.69% for direct prompting with the same Qwen3-235B-A22B-Thinking backbone.
The direction is promising. The absolute result is also a warning: even the best reported result in those experiments identifies the exact step only about one-third of the time on the macro-average metric. The authors note that the method relies on LLM interpretation, can inherit missed detections across stages, and still warrants manual verification for important or high-stakes diagnoses.
This is arXiv v1, not proof of a production debugging standard. The practical value is the shift in what we ask the evidence to explain.
The Error-Lifecycle Trace
For operating agent systems, I would turn that shift into a simpler four-stage trace. This is my operator synthesis, not the TRAJDEBUG method.
The four stages are initiate, propagate, resolve or persist, and impact.
1. Initiate
Record the earliest observed deviation that could plausibly affect the outcome.
Capture the step, the decision or state change, the violated constraint or reference, and the evidence supporting the diagnosis. In the inventory example, initiation is not “the report omitted records.” It is the unsupported decision to treat a paginated response as complete.
2. Propagate
Identify which later decisions, state objects, and artifacts consumed the bad assumption.
This is where a mistake becomes operationally expensive. The incomplete snapshot propagated into calculations, the ledger, the exception list, and the report. A dependency map is more useful than a list of later wrong statements because it shows the contamination path.
3. Resolve or persist
Test whether later action actually superseded the error.
An apology, revised explanation, or acknowledgment is not resolution. The error is resolved only when superseding evidence changes the affected reference state and dependent artifacts are rebuilt, invalidated, or verified safe. Fetching the missing page without regenerating the ledger would still leave the error active in the artifact chain.
TRAJDEBUG uses a similar evidence discipline in its own method: an error is resolved only when a later step explicitly revisits the reference object and provides citable superseding evidence. For operators, the implementation rule is straightforward: correction is a state transition, not a sentence.
4. Impact
Connect the still-active error to the terminal outcome.
Name the failed acceptance condition, the affected deliverable, and the confidence of the causal explanation. Do not stop at “the run failed.” State that the incomplete source set caused eight records to be absent from both the updated ledger and the final report—if the retained evidence supports that conclusion.
A compact trace record can look like this:
step_id: tool_call_17
observed_deviation: first page treated as complete inventory
evidence_refs:
- response.record_count=72
- response.continuation_token=present
- task.expected_record_count=80
downstream_dependents:
- working_snapshot_v3
- reconciled_ledger.csv
- exception_report.md
resolution_status: persisted
terminal_impact: eight records absent from ledger and final report
review_confidence: high
The schema is not the point. The causal links are.
Retain causal breadcrumbs, not everything forever
An error-lifecycle review is only as good as the evidence the system keeps. But “log everything” is not a serious retention policy.
The useful minimum is structured causal breadcrumbs:
- task constraints and reference values;
- material assumptions and short decision summaries;
- tool inputs, outputs, pagination, and error receipts;
- state transitions and the objects they supersede;
- artifact lineage and downstream dependents;
- correction attempts and verification evidence;
- terminal acceptance results;
- confidence levels and known evidence gaps.
This does not require hidden chain-of-thought capture. It requires enough externalized evidence to reconstruct consequential decisions and state changes.
Retention should be bounded by purpose. Sensitive tool outputs may need redaction or restricted access. Credentials should never be copied into traces. High-volume observations may need summaries plus hashes or durable references. Teams should set deletion windows based on debugging value, privacy, security, storage cost, and legal obligations.
The goal is not maximal observability. It is sufficient evidence for causal review.
A practical review procedure
When a long-running run fails, review it in this order:
- Define the terminal failure. Name the unmet acceptance condition, not just the evaluator score.
- Collect candidate deviations. Include contradictions, unsupported commitments, failed tool calls, state changes, and invalid artifacts.
- Separate harmless errors. Remove deviations that were corrected before downstream use or that have no supported path to the failure.
- Trace dependencies forward. For each remaining candidate, identify the decisions and artifacts that inherited it.
- Test resolution. Look for superseding state and regenerated dependents, not verbal acknowledgment.
- Compare causal explanations. Select the earliest error necessary to the best-supported explanation. Preserve alternatives and evidence gaps.
- Name the repair target. Identify the earliest state or artifact that must be corrected and the downstream work that must be invalidated or replayed.
That last output is still diagnosis, not repair. The trace can identify where a safe recovery should begin. Choosing whether to retry, repair, escalate, or stop is a separate recovery decision. The trace cannot, by itself, choose the right action, prove that a rerun will succeed, or guarantee production reliability.
What this evidence does not prove
TRAJDEBUG’s benchmark asks for one annotated critical step, but that does not establish that every real-world failure has one unique root cause. Its results apply to the tested models, prompts, datasets, and experimental settings—not universal superiority or a safety guarantee. Its application studies turn diagnoses into prompt feedback for reruns; they do not show that tracing automatically repairs an active run.
The operator model is evidence-bounded too. A clean-looking lifecycle can still be wrong when evidence is incomplete, forged, stale, or misread. Causal reconstruction should carry confidence and competing hypotheses, especially when the counterfactual cannot be tested.
Instrument the lifecycle before the next failure
A common operating failure is discovering after the run that the decisive evidence was never retained. The tool response was summarized without its continuation token. The state snapshot was overwritten. The corrected claim was logged, but the contaminated artifact lineage was not.
Pick one consequential long-running workflow and make every material state change answer four questions.
- What initiated this state?
- What consumed it?
- What evidence superseded it?
- What terminal result did it affect?
A final score tells you that the run failed. An error-lifecycle trace tells you where the failure remained alive.
That is the difference between labeling a bad outcome and finding a defensible place to begin fixing it.