A tool call times out after an agent asks an API to create a resource. The model sees an error and tries again. The service may already have created the resource before the response disappeared.
The safe next step depends on the operation's identity, not on how confidently the agent says it will retry. Persist one identity for the intended business action, reuse it only under the endpoint's documented retry contract, and reconcile an ambiguous result before creating a new action.
This is established distributed-systems engineering applied to agents. Malcolm Featonby's AWS Builders' Library article on idempotent APIs explains the underlying ambiguity and the role of caller-provided request identifiers. The operation journal and tests below are my proposed application design, not a measured reliability result.
Keep unknown separate from failed
Consider a hypothetical provisioning agent creating a workspace. The request reaches the service, the workspace is created, and the connection closes before the client receives the response. A timeout tells the caller that it did not receive a timely result. It does not tell the caller whether the workspace exists.
AWS illustrates this problem with an EC2 launch: simply repeating an uncertain request can create multiple resources. Reconciliation also needs care because an existing resource might belong to another provisioning process. Matching its name is weaker evidence than linking it to the original request identity. Source: retrying and side effects.
My recommendation is to preserve an explicit unknown state. Do not convert a transport error into business failure in the tool wrapper. If the API supports a documented same-key retry, use that contract. Otherwise, query an authoritative receipt or resource associated with the original operation. If neither route can resolve the outcome, pause the mutation and surface the uncertainty.
A read that temporarily returns no result is not automatically proof that nothing committed. Your reconciliation policy must account for the service's visibility guarantees and the possibility that the original request is still running.
Give intent an identity before the network call
An idempotency key identifies an intended action within a service's supported scope. It is not a general authorization token, and adding a header to an arbitrary API does not make that API idempotent.
AWS describes caller-provided identifiers because identical parameters can represent different intentions. Two matching requests to launch an instance might be an accidental retry, or the caller might genuinely want two identical instances. A hash of the arguments cannot decide that business question. Source: reducing client complexity.
For an agent application, I would create a durable operation journal before invoking the mutating tool. A minimal record contains the operation ID, caller or tenant, action type, target, reviewed parameter fingerprint, authorization reference, current state, and eventual service receipt. Keep sensitive arguments out of general logs; a fingerprint is useful for comparing parameters, not for deciding whether two intentions are the same.
If the agent retries the same approved operation, the executor loads that record and preserves its identity. If the requested target or parameters change, the executor rejects silent reuse and sends the new intent through the applicable approval path. A genuine second workspace needs its own identity even when every parameter matches the first.
The journal does not replace the server's guarantees. AWS explicitly calls out atomic recording of the request token with the related mutation. A client-side record cannot force an external service to make those writes atomic. For endpoints without that support, reconciliation and bounded manual resolution remain part of the design.
Test the gap between commit and receipt
A happy-path test proves little about this failure mode. The interesting boundary is the interval after the service commits but before the client records its receipt.
Use a sandbox or an isolated fake service with a durable ledger. These are proposed tests, not reported results:
- Drop the response after the service commits. Resume the same journaled operation and verify that the documented retry or reconciliation route identifies the original resource rather than creating another.
- Crash the client after receiving success but before recording it locally. Restart from durable state and check that recovery retains the operation identity.
- Submit the same identity with changed parameters. Verify that the wrapper or service rejects the mismatch instead of quietly changing the business action.
- Submit a genuinely new intent with identical parameters. Verify that a new operation identity permits the second authorized action.
- Exercise expiry and concurrency. Retry near the service's retention boundary and issue overlapping attempts, then inspect authoritative side effects rather than counting successful tool responses.
The endpoint's actual contract controls the expected result. Record its key scope, retention interval, parameter-mismatch rules, concurrent-request behavior, and available lookup mechanism. Do not inherit those assumptions from a different API or imply an unlimited exactly-once guarantee.
AWS also distinguishes a replay that merely avoids another side effect from a replay that returns a semantically equivalent result. Returning only “already exists” can still leave the caller unsure whether this operation created the resource. A useful tool receipt links the resource to the original operation and gives the agent enough information to continue without guessing. Source: retries and semantic equivalence.
Put recovery policy in the executor
The model can propose the next step, but the executor should enforce whether it is a retry, a reconciliation read, or a newly authorized action. Natural-language phrasing should not silently change that classification.
This extends the distinction in Build the Workflow Before the Prompt: a better instruction cannot substitute for durable runtime state. It also belongs beside the resource controls in Parallel Tool Calls Need a Resource Budget. Parallel attempts can consume resources and duplicate side effects if the executor treats every invocation as a fresh action.
Start with one mutating tool. Write down its retry contract, add a durable intent record, and inject the lost-response failure in a sandbox. Inspect the service ledger afterward. If you cannot explain whether recovery continued the original action or created a second one, keep that tool out of unattended retry loops until you can.