Skip to content
← All articles
6 min read

API idempotency and retries: building reliable network integrations

Network failures and client timeouts make duplicate calls inevitable. Applying API idempotency ensures that an operation can be retried safely multiple times without creating duplicate records or executing unintended side effects.

Why network timeouts cause duplicate operations

When a client sends a request to create a payment or submit an order, the server may process the record successfully, but the network connection may drop before the response reaches the caller. If the client assumes failure and sends another request, a standard backend creates a second record.

Network communication is inherently unreliable. Systems that integrate external APIs must assume that any outgoing request can succeed on the receiver while timing out on the caller.

Managing state with the Idempotency-Key header

The standard mechanism for handling unsafe HTTP methods like POST is passing a unique token, typically via an Idempotency-Key header. Before dispatching the request, the client generates a unique identifier, such as a UUID, and attaches it to the payload metadata.

When the server receives a request, it checks whether that key has already been processed. If a saved execution result exists for that key and payload hash, the server immediately returns the cached response instead of running the business logic again.

Enforcing deduplication with database constraints

In-memory caches and distributed locks reduce load, but the definitive barrier against duplicates is a unique constraint at the database tier. Storing the idempotency key or natural business identifiers in a table with a unique index prevents race conditions when concurrent requests arrive simultaneously.

If two worker threads attempt to insert the same operation concurrently, one transaction commits while the other triggers a unique constraint violation. The application intercepts this violation, waits for the primary transaction to finish, and returns the established record.

  • Define a unique index on the combination of the client identifier and the idempotency key.
  • Persist incoming request status as pending, succeeded, or failed.
  • Store the full response body and HTTP status code alongside the key to return identical responses.

Processing incoming webhook retries safely

External platforms such as payment gateways notify your system of state updates through webhooks. If your endpoint responds with a server error or does not answer within the timeout threshold, the provider queues the webhook for automatic redelivery.

Webhook receivers must be idempotent by design. Each incoming event payload includes an event identifier or an entity version timestamp. The receiving endpoint should record processed event identifiers and ignore duplicates without returning an error code to the sender.

Retrying failed calls with exponential backoff and jitter

When a service receives temporary rate-limit or service-unavailable errors, immediate retries can overwhelm an already struggling API. A resilient integration uses exponential backoff: each subsequent retry interval grows progressively longer.

To prevent multiple worker processes from retrying at identical intervals and creating a thundering herd spike, algorithms add jitter—a random variation applied to each delay window. This spreads outbound requests evenly across the recovery period.

A step-by-step lifecycle for order creation

To prevent duplicate orders, the client generates a unique idempotency key and initiates the POST call. The server begins a database transaction, checks for an existing record with that key, and if absent, creates the order and stores the idempotency record atomically.

If an identical key arrives while the initial operation is still running, the server rejects or holds the second call until the first concludes. If the first operation finished, the server serves the stored response directly from the database without touching order creation logic.

  • Client generates a distinct UUID for the checkout attempt.
  • Server attempts an atomic insert into an idempotency table or applies a row-level lock.
  • Database executes order placement and updates the key status to completed.
  • Client retries on network timeout using the original UUID and receives the existing order confirmation.
Questions

FAQ

Didn't find an answer — ask on the call or in Telegram.

How long should idempotency keys be retained in storage?

Retention windows depend on the business domain and dispute resolution periods. Systems retain keys until the window for automatic client retries and network recovery has passed, after which the key expires and any subsequent request with that identifier is treated as a new operation.

What should happen if the same key is sent with a different payload?

The server should compute a hash of the incoming request body and compare it with the stored hash for that key. If the key matches but the parameters differ, the server must return an HTTP 422 Unprocessable Entity or HTTP 409 Conflict error, indicating an idempotency conflict.

Are GET and DELETE requests naturally idempotent?

According to HTTP specifications, GET, PUT, and DELETE methods are idempotent by definition. Repeated GET requests retrieve the current state without side effects, and multiple DELETE requests result in the resource being absent, even if subsequent calls return an HTTP 404 status code.

Related service

CRM, ERP and API integrations built for your workflows

When teams copy data between systems by hand, delays and mistakes become part of the process. We automate the exchange and make its status visible to the people running it.

Learn about the service
Other articles
Contact

Shall we discuss your task?

Thirty minutes on a call — and you understand the architecture, timeline and engagement format. No obligations.

Or message us on Telegram →Follow our Telegram channel →

Enter your Telegram or email so we can reply.

We reply within 15 minutes.