Operating manual
Tag: evaluation
A focused reading path for evaluation: related field notes, evidence trails, and operating questions from the archive.
- population-monitoring-as-an-agent-warning-signal.md
Population Monitoring as an Agent Warning Signal
Individual agent checks can miss collective drift. Here's how to evaluate a population-level warning signal without mistaking it for a safety guarantee.
open artifact → - what-can-you-remove-after-harness-distillation.md
What Can You Remove After Harness Distillation?
Harness distillation can make some runtime guidance removable. Use a retain-or-retire map to separate learned habits, tool interfaces, and enforced controls.
open artifact → - scoring-definition-is-the-lever-in-agent-evals.md
Scoring Definition Is the Lever in Agent Evals
A 24.0%-57.0% consensus swing on identical held-out data is a property of the scoring definition, not the agents. Lock the definition before reading the number.
open artifact → - spreadsheet-rag-can-find-the-right-wrong-chunk.md
Spreadsheet RAG Can Find the Right Wrong Chunk
A spreadsheet retrieval hit can lose the headers that give a value meaning. Separate missing evidence, missing context, and incorrect answers.
open artifact → - competent-agents-can-still-fail-to-coordinate.md
Competent Agents Can Still Fail to Coordinate
Agent capability is not proof of a valid shared commitment. What scheduling experiments suggest builders should test across different owners.
open artifact → - build-the-workflow-before-the-prompt.md
Build the Workflow Before the Prompt
Build verified tool-use examples from executable workflows. What ToolGrad shows, what its pass rate does not, and how to keep challenge cases separate.
open artifact →