An idempotency key is a contract, not a deduplication cache

The idea

Networks fail ambiguously. A client sends POST /charges, the connection drops, and the client cannot tell whether the charge happened. Retrying risks charging twice. Not retrying risks losing the order. Neither choice is acceptable when the operation moves money or sends a message to a person.

An idempotency key resolves the ambiguity. The client generates a unique key for each logical operation and sends it with the request, usually in an Idempotency-Key header. The server records the key alongside the outcome. If the same key arrives again, the server returns the recorded result instead of doing the work a second time. The client gets at-least-once delivery, and the system gets a single effect.

An example

A checkout page generates a key when the user presses pay and keeps it until the payment resolves.

• The client sends POST /charges with key k1 and an amount.
• The server inserts a row (tenant, k1, request_hash, status=in_progress) under a unique constraint, in the same database transaction that will create the charge.
• The charge is created, the response body and status code are stored on the row, the status becomes completed, and the transaction commits.
• The response is lost on the way back. The client retries with k1.
• The server finds the completed row, checks that the request hash matches, and returns the stored response. No second charge exists.

If the retry arrives while the first attempt is still running, the unique constraint catches it, and the server answers with a conflict that tells the client to try again shortly.

Where it breaks

The common failure is storing the key somewhere other than the effect. If the charge commits in one place and the key is recorded in another, a crash between the two produces exactly the duplicate the key was meant to prevent.

Other ways it goes wrong:

• The client generates a fresh key per attempt rather than per intent. Every retry looks new.
• The same key arrives with a different body. Replaying the old response hides a client bug. Reject it.
• A retry arrives after the key has expired, and the work runs again.
• The handler calls a third party that is not protected by your key. Your record is clean, the downstream system holds two copies.

It is also the wrong tool when the operation is already idempotent, such as a full replacement PUT, or when a natural business identifier like an order number can carry a unique constraint directly. I prefer that constraint when it exists.

In practice

• Require keys on create endpoints that move money, send messages or provision resources. Optional keys get skipped.
• Scope keys per tenant or credential so two clients cannot collide.
• Store a hash of the normalized request and reject mismatches with a clear error.
• Insert the key first, under a unique constraint, inside the transaction that performs the write.
• Store final outcomes, including validation errors. Do not store transient server errors, so a retry can still succeed.
• Derive a key for each downstream call and pass it through, so the guarantee extends past your boundary.
• Document the retention window and mark replayed responses with a header.
• Test it by killing the process between steps and firing concurrent duplicates, not only by sending the same request twice in sequence.

An idempotency key only protects you if it is recorded in the same transaction as the effect it guards and is bound to the exact request that created it.