m@berryhill: ~/berryhill.dev/posts/ai-benchmarks-must-name-the-serving-route.md
~ homeposts/about.md
m@berryhill in ~/posts$ cat ai-benchmarks-must-name-the-serving-route.md
---
title:  AI Benchmarks Must Name the Serving Route
date:   2026-09-11
topic:  ai
read:   10 min
words:  2,163
slug:   ai-benchmarks-must-name-the-serving-route
views:  live post
tags:   [ai, evaluation, infrastructure]
---
essay · long read

AI Benchmarks Must Name the Serving Route

Before transferring an AI benchmark score to your deployment, record the serving configuration, capability coverage, and failures behind it.

table of contents
  1. The model name is not the tested system
  2. Give every result a configuration record
  3. Establish capability coverage before scoring
  4. Keep admitted failures in the result
  5. What the preprint can—and cannot—support
  6. Ask for this record before transferring a score
  7. Source

Consider a hypothetical deployment review: an evaluation lead gets a benchmark result. The model name matches the deployment proposal. The endpoint, precision, tool-call parser and retry policy are missing.

Would you treat those as the same tested system?

A benchmark score should travel with a configuration record. A serving route is the path and configuration through which you access a model: the endpoint, serving software and request handling between your application and its answers. The model name alone does not identify that system.

My recommendation: before using a benchmark to choose a deployment, ask what was tested, which requests it could accept, and what happened to the attempts that failed. Those are separate questions. A useful evaluation record answers all three.

The model name is not the tested system

The arXiv preprint IB2: A Protocol for Measuring Enterprise AI Systems by Serving Route, Not Model Identifier proposes making this distinction explicit. Its system identity includes the model revision, serving route, precision, reasoning effort, observed request envelope, tools, output contract and harness.

That definition gives an evaluation lead something more useful than an argument about which model is best: a way to ask whether a result describes the configuration under consideration.

Suppose a benchmark used a tool-call parser different from the one behind your endpoint. Even if the model identifier matches, the comparison has an unresolved field. That does not prove your deployment will perform worse. It means the benchmark has not answered that deployment question on its own.

The practical distinction is between choosing what behavior to test and identifying the system whose behavior you measured. You need both. The same configuration discipline matters when a model upgrade changes an agent’s memory behavior: keeping one component’s name or stored records intact does not establish continuity of the whole system.

The model name leaves fields openFour connected groups identify model revision, route and precision, request contract and tools, and harness and parser.A tested system has four partsA model name leaves the rest unresolved.Model and revisionIdentifier, build, revision hash.Endpoint and serving routeURL, serving software + version, precision,reasoning.Request contract and toolsPrompt envelope, tools + selection, outputcontract.Harness and parserHarness version, parser/scorer, retry +fallback policy.
A matching model name leaves the rest of the tested configuration unresolved.

Give every result a configuration record

Here is the record I would request alongside a score. It extends the preprint's system-identity proposal into a deployment comparison aid; it is an operator recommendation, not a validated procurement standard.

Field Tested configuration Proposed deployment
Model and revision Exact identifier or unknown Exact identifier or unknown
Endpoint and serving route Endpoint, access mode, serving software/version where disclosed Same fields
Precision Reported precision or unknown Reported precision or unknown
Reasoning settings Effort, budgets and limits Intended settings
Request envelope Prompts, context, sampling, token and timeout limits Intended request configuration
Tools and tool choice Schemas, available tools, selection policy Required tools and policy
Output contract Required schema and formatting rules Application requirements
Harness and parser Versions and response parsing behavior Versions and response parsing behavior
Retry and fallback policy Allowed retries, fallback routes and accounting Intended policies
Evaluation date When this configuration was tested Date of the deployment comparison

Write unknown when a field is unavailable. Do not fill it from a model announcement, an assumed provider default or a different endpoint's documentation.

Unknown is a limit on interpretation, not evidence of poor performance. It tells you where to ask for disclosure or run a test on the intended route.

Coverage first; failures stay visibleA required contract branches into unsupported coverage or admitted execution; admitted attempts retain usable responses and failures.Coverage first; failures stay visibleTwo attempts, two questions, two denominators.Required request contractCapabilities the workload actuallydemands.UnsupportedCoverage gapStays outsidethe admitteddenominator.Re-run on aroute thatsupports it.Admitted runAttempts countedUsable responsesand failuresretained.Denominatorincludes both.
Unsupported before admission and failed after admission answer different questions.

Establish capability coverage before scoring

First declare the request contract. Then establish whether the endpoint supports it. Only then admit it to the corresponding evaluation.

For example, a workload may require a particular tool interface and structured output. An endpoint that cannot accept that contract has a capability-coverage problem for this workload. Calling that a zero score for model ability would collapse two different questions.

The IB2 protocol distinguishes pre-admission binding from admitted execution in sections 4.2 and 4.5. That distinction is useful, but its implementation is not a guarantee: the authors disclose that the PartialCoverage branch was unreachable in the reported implementation, and that the binding gate missed strict-JSON adherence failures after extended reasoning.

Keep these outcomes separate:

Outcome What to report
Required capability unsupported before admission The unsupported requirement and coverage/binding outcome; do not automatically label it zero model ability
Contract accepted; execution later fails An admitted attempt, its failure category, and its treatment in each metric
Parseable response supplies some work products Assertion credit under the declared rubric, alongside contract validity and failure status
No parseable response The absence and its scoring treatment; the IB2 rubric assigns no assertion credit when there is nothing parseable to score

For a deployment decision, unsupported capability can still be disqualifying. You do not need to relabel it as low intelligence to decide that an endpoint cannot serve your application.

Keep admitted failures in the result

Once an attempt has been admitted, do not let it disappear because the response was inconvenient to score.

Report admitted attempts, usable responses, failure categories and the denominator for every metric. If you report answer quality only among usable responses, label that condition and put failure-inclusive results beside it. A reader should not have to infer which attempts were excluded.

There is an important scoring nuance here. Under IB2's section 4.5 rubric, retaining failures does not mean assigning zero to every contract-invalid response. Parseable responses can receive assertion credit for work products actually supplied. Failure class itself does not add or subtract points. An absent parseable response earns zero mechanically because there is no supplied work product to credit.

In your own evaluation, define those rules before comparing results. Keep capability coverage, contract validity, usable-response rate and answer quality distinguishable. They answer different operational questions.

What the preprint can—and cannot—support

The value I take from IB2 is the proposed evaluation contract, not a vendor ranking.

Its evidence has substantial limits. The preprint's limitations and disclosure sections describe a sealed synthetic task bank and controlled tool sandbox, not a measured population of production workloads. There is no human baseline. Each configuration was run once; task-resampling intervals do not establish run-to-run variance.

As the authors disclose in the versioned preprint, access mode, evaluator harness generation and the serving tool-call parser change together in the serving-arm comparison. That comparison cannot isolate the effect of the route, engine, precision or provider. The earlier calibration runs were retrospectively audited against the finalized binding gate; only the replacement route was prospectively bound under the complete gate. Consumed reserve, retry-audit asymmetry and fragile nominal tier boundaries further constrain ranking interpretations.

All authors work for Iterate.ai. The instrument was built against Iterate systems, although no Iterate system appears in the reported cohort. The corpus and raw run evidence remain private, and one route is intentionally unnamed and under-identified. This article does not independently replicate those experiments or identify that route.

The authors also say they found no documented instance of public benchmarks misattributing route limits to model weights. The argument is therefore not that this preprint proves a market-wide reporting failure. It offers a protocol for making configuration and scoring boundaries explicit.

Before transferring a scoreFour review steps compare configurations, disclose unknowns, inspect coverage and failure denominators, and test material differences.Before transferring a scoreFour review steps before transferringa score.1. Compare configuration identityModel, route, precision, tools, outputcontract.2. Disclose unknowns explicitlyMark every unknown field as unknown; donot infer.3. Inspect coverage and failuredenominatorsUnsupported vs admitted; failures stayvisible.4. Test material differencesIf anything differs, evaluate the intendedroute.
Matching fields makes comparison inspectable; it does not prove workload transfer.

Ask for this record before transferring a score

Copy the configuration table above into your next evaluation review, then append this results record:

Workload and required request contract:
Capabilities supported / unsupported / unknown:
Admission decision and evidence:
Admitted attempts:
Usable responses and definition of usable:
Failure categories and counts:
Contract-valid responses:
Assertion-credit rubric and treatment of invalid responses:
Failure-inclusive metrics and denominators:
Conditional answer-quality metrics and denominators:
Retries and fallbacks included / excluded:
Evidence links and inaccessible evidence:
Differences from the proposed deployment:
Unknown fields and unresolved questions:

Matching the fields does not prove that a score transfers to your workload. It makes the comparison inspectable. Material differences are a reason to evaluate the intended configuration, not to quietly borrow the old score.

Before the next model-selection meeting, ask for this record alongside the leaderboard. You can still use the benchmark. You will be clearer about which decision it supports—and which decision still needs a test.

Source

IB2: A Protocol for Measuring Enterprise AI Systems by Serving Route, Not Model Identifier. Versioned arXiv preprint. Protocol and scoring definitions: sections 3 and 4; limitations and disclosures: sections 10 and 11. Recommendations in this article are operator analysis, not independent replication.

m@berryhill in ~/posts$
$ cd ../ · back to posts/