A team records the random seed for an agent simulation. Six months later, someone tries to rerun it.
The seed is there. The simulation is not.
The scheduler changed. One behavior function has no recorded version. A dataset moved. A proprietary dependency is unavailable. The LLM endpoint still has the same marketing name, but not necessarily the same model behind it. The original validation said the output “looked right,” with no declared comparison method or tolerance.
This is not a seed-management problem. It is an underspecified-system problem.
Here, “agent simulation” includes rule-based, agent-based, and LLM-backed systems in which autonomous entities act inside a modeled environment.
An agent simulation is reproducible only to the extent that another operator can reconstruct the system that consumed the seed, inspect its assumptions before execution, and compare the rerun against explicit criteria afterward. The operating requirement is not more prose. It is a versioned, machine-checkable reproducibility contract.
A seed is not a simulation
A seed controls a source of randomness. It does not define:
- which entities and state variables exist;
- how state is initialized;
- which rules change that state;
- when and in what order those rules run;
- which datasets, libraries, services, and hardware assumptions the model depends on;
- which model, prompt, and inference settings shape an LLM-backed decision;
- or what counts as a successful reproduction.
Even the phrase “same seed” can hide ambiguity. Which random-number generator used it? Was there one generator or several? Did every agent share a stream? Was state reseeded between episodes? Did asynchronous execution change the order in which random draws were consumed?
The seed matters. It is one field in a much larger contract.
The practical move is to treat the model description as a build artifact: version it, validate it before execution, resolve it into the exact conditions used for the run, and preserve that resolved artifact with the results.
Narrative documentation leaves executable ambiguity
Prose is useful. It can explain why a model exists, what its abstractions mean, and which assumptions deserve human scrutiny. But prose is a weak place to discover a missing reference five minutes before a rerun.
A person may infer that a behavior called move should appear in the schedule. A validator can check whether it actually does. A person may assume that an output metric refers to a declared variable. A validator can reject the contract when it does not. A person may skim past a dataset mentioned in one paragraph and omitted from the environment instructions. A machine-readable reference can fail visibly.
A July 30, 2026 arXiv v1 paper by Zhou He offers a useful, bounded example. VISA: A Structured Description Protocol for Agent-Based Simulation Models Towards Machine Reproducibility proposes a symbol-based protocol built from eight interconnected tables: Agent, Variable, Sensing, Internal Function, Associated Data, Input/Output, Schedule, and Validation.
The full paper also defines nineteen consistency rules—four within individual tables and fifteen across tables—and three reusable skills for authoring, checking, and code generation. Examples include checking function-ID uniqueness, confirming that declared functions appear in the execution schedule, verifying that validation objects exist, and resolving references to associated data.
Those checks turn documentation into inspectable input for execution.
VISA is a proposal, not a consensus standard, and its evaluation does not establish that it outperforms narrative descriptions across the field. The paper itself says controlled comparison is still needed. But it demonstrates the shape of a better question: not “Did we write enough?” but “Can the description reject its own omissions?”
The Six-Surface Agent Simulation Reproducibility Contract
For an operating team, I would organize that question into six surfaces. This is a working implementation pattern, not VISA’s eight-table schema.
1. Model structure and state
Define the entities, state variables, initial conditions, relationships, units, and invariants that make the model what it is.
A machine check should be able to answer: Are all required fields present? Do types and ranges match? Do references resolve? Are units explicit? Can the initial state violate a declared invariant?
Preserve the schema version, normalized contract hash, initialization manifest, and validation result with the run.
2. Behavior and update logic
Define the rules, transitions, decision functions, ordering assumptions, and termination conditions that change state.
A machine check should confirm that every referenced behavior exists, declared inputs and outputs are compatible, and transition and termination references are complete. Human review still owns the harder question: whether those rules faithfully represent the intended phenomenon.
Preserve the rule-set version, executable mapping, validation result, and any semantic assumptions that remain outside automated checks.
3. Schedule and randomness
Define when agents and environments update, in which order, and where randomness enters.
The contract should name the scheduler, event order, timestep semantics, random-number generators, seed values, stream allocation, and reseeding behavior. A seed without these fields can produce the appearance of control while leaving the execution path ambiguous.
Preserve the scheduler version, random-state manifest, seeds, run count, and a variance summary.
4. Environment, data, and dependencies
Define the datasets, libraries, services, hardware assumptions, and external resources required to execute the model.
Dependencies should resolve to versions or immutable references. Data should carry provenance. External services should have declared availability checks. If a proprietary package or missing dataset prevents reconstruction, the contract should fail with that blocker instead of allowing a vague “the model did not reproduce” verdict.
I recommend preserving a dependency lock or image digest, data provenance, environment fingerprint, availability results, and unresolved dependency constraints.
5. LLM runtime configuration
Define the inference conditions behind every LLM-backed agent decision.
Zhou He’s paper names four necessary fields: exact model version, prompt, decoding parameters, and seed. In an operating contract, I would also preserve the resolved provider identifier, tool configuration, and invocation metadata when they affect behavior.
These fields narrow ambiguity. They do not freeze a vendor’s backend, preserve a model forever, or guarantee identical output from stochastic inference.
6. Validation and reproduction receipts
Define what will be checked before execution, what outputs will be compared afterward, which tolerances apply, and which blocker classes are possible.
Structural checks should pass before compute begins. Expected outputs, comparison methods, tolerances, and qualitative criteria should be declared before seeing the rerun. The result should be a reproduction receipt containing check results, output comparisons, tolerance decisions, drift disclosures, unavailable dependencies, and a bounded verdict.
I treat the receipt as evidence about a reproduction attempt, not as a certificate of scientific truth.
Make the contract fail before the run does
My preflight rule is simple: discover that a scheduled function does not exist before the simulation starts.
The same applies to an output that references an undeclared variable, a validation rule with no target, an unavailable dataset, an unresolvable library version, an unnamed scheduler, or an LLM call with no recorded model version.
A useful preflight gate should reject at least five classes of failure:
- Schema failure: required fields, types, or ranges are missing or invalid.
- Reference failure: a rule, variable, dataset, output, or validation target points to something undeclared.
- Execution failure: scheduling, ordering, termination, or randomness semantics are incomplete.
- Dependency failure: a required package, dataset, model, or service cannot be resolved under the declared conditions.
- Comparison failure: the contract does not say what outputs matter or how a rerun will be judged.
This does not make the simulation correct. It makes preventable ambiguity expensive earlier, when it is still cheap to fix.
Structural consistency is not behavioral truth
Machine-checkable completeness can answer whether the pieces connect. It cannot answer whether the model is a faithful representation of the world.
That boundary is easy to lose because “executable” sounds stronger than it is.
A model can have complete references, valid types, a fully specified schedule, and reproducible code while encoding a bad causal assumption. It can reproduce a qualitative pattern for the wrong reason. It can satisfy a tolerance chosen after the result was seen. It can be internally consistent and scientifically weak.
The VISA evaluation makes the distinction concrete. Its third case passed all nineteen structural rules, yet the output could not be reproduced. Structural validity cleared one gate. Dependency availability blocked the next.
Use automated checks to eliminate omissions and contradictions. Keep behavioral fidelity, causal validity, semantic meaning, and scientific judgment as separate review surfaces.
The blocked reproduction is part of the result
The paper’s evaluation applies an author-check-code loop to three external agent-based models.
Two NetLogo models were reimplemented in Python and recovered qualitative signatures rather than bit-exact trajectories. The Rebellion case reproduced its reported punctuated-equilibrium pattern under the paper’s validation criteria. The Wolf Sheep case recovered several qualitative dynamics, but one criterion was reported as directional rather than an unqualified pass.
The third model, built in AnyLogic, was captured structurally but not reproduced. Its movement depended on a proprietary Pedestrian Library, and the backing data was unavailable, according to the reported case analysis.
That result matters because the contract does not need to recreate an unavailable proprietary library to be useful. It needs to localize and preserve the blocker, preventing dependency failure from being misreported as model failure. Blocked is a legitimate reproduction state when the evidence is explicit.
LLM configuration capture does not freeze the provider
LLM-backed simulations add a harder boundary.
Recording the exact model version, prompt, decoding parameters, and seed is necessary. It gives the next operator a much narrower target than “we used the current version of Model X” and exposes prompt or decoding changes that would otherwise remain buried inside an application.
But no contract can guarantee permanent access to the same black-box service. Providers can update model weights, routing, safety layers, tool behavior, or infrastructure. Model identifiers can drift in meaning. Inference can remain stochastic even when the visible configuration matches. The paper states this limit directly: model-version drift and stochastic inference remain reproducibility hazards that a description protocol cannot fully eliminate.
Keep configuration capture, but stop promising guarantees it cannot deliver. Version the fields, preserve invocation evidence, run repeated trials where variance matters, disclose when exact model access is unavailable, compare behavior under declared criteria, and mark provider or model drift in the receipt. Capture narrows the uncertainty. The receipt records what survived.
End every attempt with a reproduction receipt
“Reproducible” is too often used as a property of the original artifact. Operationally, it should be a claim about an attempted reconstruction with evidence attached.
A reproduction receipt should answer:
- Which contract version was resolved?
- Which structural checks passed or failed?
- Which code, data, dependencies, models, and services were available?
- Which expected outputs were compared?
- Which comparison methods and tolerances were used?
- Which behaviors matched fully, directionally, or not at all?
- Which differences may be explained by randomness, model drift, environment drift, or implementation changes?
- Which conditions remained blocked or unverifiable?
I recommend more than two verdict states. A useful set might include:
- reproduced within declared criteria;
- partially reproduced;
- directionally reproduced;
- not reproduced;
- blocked by unavailable dependency;
- incomparable because the contract is incomplete.
Those labels do not remove judgment. Their purpose is to make the judgment name its evidence.
The resolved contract and the receipt should travel together. One describes the system the operator intended to reconstruct. The other records what happened when someone tried.
Adopt the contract one simulation at a time
Teams do not need to wait for a universal standard to make the next run more reconstructable.
Start with one simulation that matters:
- Inventory the six surfaces as they exist today.
- Convert implicit assumptions into versioned fields and resolvable references.
- Add structural checks for missing fields, broken references, scheduling gaps, dependencies, and validation targets.
- Resolve and preserve the exact contract used for the run.
- Execute the rerun under declared comparison criteria.
- Issue a reproduction receipt that records matches, drift, blockers, and uncertainty.
Then improve the contract from the failures it exposes.
The goal is not bitwise identity at any cost. It is a more honest engineering claim: another operator can see what was specified, what was available, what was checked, what matched, and what could not be reconstructed.
A seed is part of that record.
The contract is what makes it a system.