Idempotency & Deduplication
On this page
A customer taps "Pay 79 dollars". The payment service charges the card and saves the charge, but the response is lost on a flaky mobile connection. The app sees a timeout and does what any sensible client does: it retries. The service receives what looks like a brand-new payment and charges the card again. The customer has now paid 158 dollars for one order, and support has a refund to process.
Retries are unavoidable in distributed systems: networks drop responses, queues redeliver messages, users double-click. The question is not how to prevent duplicates from arriving, but how to make processing a duplicate harmless. An operation that has the same effect no matter how many times it runs is idempotent.
How it works
Why duplicates happen

- Client retries after a timeout, a dropped connection, or a 502 from a load balancer.
- Double submits from impatient users.
- Queue redelivery: with at-least-once delivery, a consumer that crashes before acking gets the message again.
- Replays: reprocessing a stream from an earlier offset after a bug fix.
From the client's side, "no response" is ambiguous: the server might have done nothing, done everything, or be in the middle. The only safe answer is to make retrying harmless.
Some operations are naturally idempotent
PUT /users/42/email {"email": "a@example.com"}: setting a value twice leaves the same value.DELETE /carts/42/items/7: deleting twice leaves it deleted (return 204 or 404, both fine).UPDATE … SET status = 'SHIPPED' WHERE id = 7 AND status = 'PACKED': a conditional state transition happens once.
Others are not: POST /charges, "add $10 to balance", "send email", "increment counter". These need explicit protection.
Idempotency keys

- The client generates a unique key (a UUID) once per user intent, not per attempt, and sends it as
Idempotency-Key: 9f1c…. Every retry of that payment reuses the same key. - The server tries to insert the key into an
idempotency_keystable with a unique constraint, statusin_progress. - New key: do the work, then store the response (status code and body) with the key. Ideally the charge row and the key row are written in the same database transaction, so you can never have one without the other.
- Key exists and finished: return the stored response. No second charge.
- Key exists, still in progress (the first attempt is running): return
409 Conflictand let the client retry shortly. - Keys expire after a window (often 24 hours), long enough to cover any realistic retry.
Also store a hash of the request body with the key. If a client reuses a key with a different amount, reject it as a client bug instead of returning the old result.
Deduplication in consumers
For queue consumers, the "key" is the message ID or, better, a business ID:
- Unique constraint: insert
processed_events(event_id)in the same transaction as the effect; a duplicate violates the constraint and is skipped. - Natural keys:
INSERT INTO shipments (order_id, …)withorder_idunique means a repeated "order paid" event cannot create two shipments. - Version or sequence checks: apply an update only if its version is newer than what is stored, which also handles out-of-order delivery.
Trade-offs and costs
- Storage: one row per key for the retention window. At 1,000 payments per second and 24 hours, about 86 million rows: fine with a TTL or daily partitions.
- An extra write per request on the protected path, usually in the same transaction, so a few milliseconds.
- External side effects (an email provider, a card network) are outside your transaction. Pass your idempotency key on to providers that accept one, or record "sent" before sending and accept a rare duplicate email.
- Window choice: too short and a slow retry slips through; too long and storage grows.
- Key scope: keys must be scoped per client or account so two customers can't collide.
When to reach for it
- Any endpoint that moves money, creates orders, books inventory or sends messages.
- Every message consumer in an at-least-once system.
- Webhook receivers: providers retry webhooks, often many times.
- Batch jobs that may be re-run after a partial failure.
Example
Checkout payment. Assumptions: 500 payments per second at peak, clients retry up to 3 times with backoff over about 30 seconds, the card processor accepts its own idempotency key.
- Client: the app creates
key = uuid4()when the user taps Pay and keeps it until it gets a final answer, through retries and app restarts. - Server:
BEGIN; INSERT INTO idempotency_keys (account_id, key, request_hash, status) VALUES (…, 'in_progress'). On a unique violation, read the row: finished → return the stored response; in progress → 409. - Processor call: pass the same key to the card processor, so even if our transaction fails after the charge, a retry does not charge twice on their side.
- Commit: write the charge row and update the key row to
donewith the response body in one transaction. - Downstream: a
payment-succeededevent goes out through an outbox. The shipping consumer insertsshipments(order_id UNIQUE), so a redelivered event is a no-op. - Cleanup: keys are partitioned by day and partitions older than 2 days are dropped.
Common mistakes
- Generating a new key on each retry, which defeats the purpose.
- Checking "does this key exist?" then inserting, a race where two concurrent retries both pass. Use a unique constraint.
- Storing the key in a different transaction from the side effect, so a crash leaves one without the other.
- Assuming the queue gives exactly-once delivery and skipping dedup in consumers.
- Treating a timeout as a failure and showing "payment failed" when it may have succeeded.
Say this in the interview
"Clients retry on timeouts, so the charge endpoint takes an idempotency key that the app creates once per tap. The server inserts the key under a unique constraint in the same transaction as the charge, and stores the response. A retry with the same key gets the stored response. We pass the key to the card processor too. Downstream consumers are at-least-once, so the shipments table has a unique order_id and duplicates are no-ops."
Used in
- Message Queue, Optimistic, Pessimistic & Distributed Locking, Write-heavy Systems.
- Design an Online Banking System, Design an Online Donation System, Design an Ad Click Aggregator.