Açıklaması şöyle
You typically implement idempotency like this:
- Check if request already processed (via key / timestamp / PK)
- If not → write data
- If yes → skip
Eğer check işlemi atomic değilse problem oluyor.
Failure Mode 1: The TTL Expiry Trap
Açıklaması şöyle
The most common idempotency implementation stores a request key with a time-to-live (TTL) — typically 24 or 48 hours. The assumption is that any duplicate will arrive within that window. In practice, this assumption frequently breaks.
Açıklaması şöyle
The fix: Never use TTL-only idempotency for operations with unbounded retry windows. Instead, use a database-backed idempotency store with a three-state model (IN_PROGRESS, COMPLETED, FAILED) where the expires_at column drives a cleanup job for storage management — not correctness. The cleanup window should be set significantly longer than your worst-case replay window (7 days minimum for Kafka-based systems).Failure Mode 2: The Partial Execution Ghost
Açıklaması şöyle
A request arrives, the system writes the idempotency key with status IN_PROGRESS, begins processing, writes half the data, and crashes — JVM OOM, container eviction, network partition. The idempotency key is now in IN_PROGRESS state. When the retry arrives, the system faces an impossible decision: did the original operation complete or not?
Açıklaması şöyle
The fix: Wrap both the business logic and the idempotency state transition in a single database transaction. If the transaction rolls back, both the business data and the idempotency status roll back together. For stale IN_PROGRESS keys (where the original processor is likely dead), use a configurable timeout threshold to reclaim and re-execute safely.
Failure Mode 3: The Concurrent Check Race
Burada check koşulu atomic değil. Açıklaması şöyle
The fix: Use INSERT ... ON CONFLICT DO NOTHING (PostgreSQL 9.5+) to make the check-and-claim atomic. If the RETURNING clause yields no rows, the key already existed — fetch its status with SELECT ... FOR UPDATE. For non-blocking behavior, SELECT ... FOR UPDATE SKIP LOCKED lets the second instance return 409 Conflict immediately rather than waiting.
Failure Mode 4: The Layer Mismatch
Açıklaması şöyle
The fix: Propagate a correlation ID from the original request as a Kafka header, and have every downstream consumer enforce its own idempotency barrier using that ID as the deduplication key.
Spring Boot + SQL Server
Kod şöyle. Burada
- Partial Execution tek transaction ile çözülüyor.
- The Concurrent Check Race, DuplicateKeyException ile çözülüyor. Eğer Postgres kullanıyor olsaydık exception yerine SQL'in kaç tane satırı değiştirdiğine bakacaktır
- The Layer Mismatch sorunu outbox pattern ile çözülüyor.
@Service@RequiredArgsConstructorpublic class IdempotentService {private final JdbcTemplate jdbc;public record Response(String result) {}@Transactionalpublic Response handleRequest(String idempotencyKey, String payload) {try {// Attempt barrier insert (atomic)// SQL Server:// INSERT INTO idempotency_table (idempotency_key, status)// VALUES (?, 'IN_PROGRESS')jdbc.update("INSERT INTO idempotency_table (idempotency_key, status) VALUES (?, 'IN_PROGRESS')",idempotencyKey);// First request owns the key → perform business logicString result = doBusinessLogic(payload);// Insert into outbox for async processing// SQL Server:// INSERT INTO outbox_table (idempotency_key, payload) VALUES (?, ?)jdbc.update("INSERT INTO outbox_table (idempotency_key, payload) VALUES (?, ?)",idempotencyKey, result);// Mark barrier as completed and store result// SQL Server:// UPDATE idempotency_table SET status='COMPLETED', response=? WHERE idempotency_key=?jdbc.update("UPDATE idempotency_table SET status='COMPLETED', response=? WHERE idempotency_key=?",result, idempotencyKey);return new Response(result);} catch (DuplicateKeyException ex) {// Barrier row already exists → handle duplicate// SQL Server:// SELECT * FROM idempotency_table WITH (UPDLOCK, ROWLOCK) WHERE idempotency_key=?IdempotencyRecord record = jdbc.queryForObject("SELECT status, response FROM idempotency_table WITH (UPDLOCK, ROWLOCK) WHERE idempotency_key=?",(rs, rowNum) -> new IdempotencyRecord(rs.getString("status"), rs.getString("response")),idempotencyKey);switch (record.status) {case "COMPLETED":// Return cached resultreturn new Response(record.response);case "IN_PROGRESS":// Someone else is working → can wait or throw 409throw new IllegalStateException("Request is already in progress");case "FAILED":// Previous attempt failed → allow retrythrow new IllegalStateException("Previous attempt failed, safe to retry");default:throw new IllegalStateException("Unknown barrier state: " + record.status);}}}private String doBusinessLogic(String payload) {// your domain logic herereturn "processed:" + payload;}private static class IdempotencyRecord {final String status;final String response;IdempotencyRecord(String status, String response) {this.status = status;this.response = response;}}}
Eğer hem SQL Server hem de Postgres için çalışsın istiyorsak şöyle yaparızz
@Service@RequiredArgsConstructorpublic class IdempotentService {
private final JdbcTemplate jdbc;
public record Response(String result) {}
@Transactional public Response handleRequest(String idempotencyKey, String payload) { boolean isWinner = false;
try { // -------------------------- // Attempt atomic barrier insert // -------------------------- // Postgres: // INSERT INTO idempotency_table (idempotency_key, status) // VALUES (?, 'IN_PROGRESS') // ON CONFLICT DO NOTHING // // SQL Server: // INSERT INTO idempotency_table (idempotency_key, status) // VALUES (?, 'IN_PROGRESS') int rows = jdbc.update( "INSERT INTO idempotency_table (idempotency_key, status) VALUES (?, 'IN_PROGRESS')", idempotencyKey );
// Postgres: rows == 1 → winner // SQL Server: INSERT succeeded → winner isWinner = rows == 1;
} catch (DuplicateKeyException ex) { // SQL Server only: duplicate → loser isWinner = false; }
if (isWinner) { // -------------------------- // Winner executes business logic // -------------------------- String result = doBusinessLogic(payload);
// Insert into outbox (side effect) // INSERT INTO outbox_table (idempotency_key, payload) VALUES (?, ?) jdbc.update( "INSERT INTO outbox_table (idempotency_key, payload) VALUES (?, ?)", idempotencyKey, result );
// Mark barrier as completed + store response // UPDATE idempotency_table SET status='COMPLETED', response=? WHERE idempotency_key=? jdbc.update( "UPDATE idempotency_table SET status='COMPLETED', response=? WHERE idempotency_key=?", result, idempotencyKey );
return new Response(result); } else { // -------------------------- // Loser reads existing row safely // -------------------------- // SQL Server: SELECT ... WITH (UPDLOCK, ROWLOCK) WHERE idempotency_key=? // Postgres: SELECT * FROM idempotency_table WHERE idempotency_key=? IdempotencyRecord record = jdbc.queryForObject( "SELECT status, response FROM idempotency_table " + (isPostgres() ? "" : "WITH (UPDLOCK, ROWLOCK) ") + "WHERE idempotency_key=?", (rs, rowNum) -> new IdempotencyRecord(rs.getString("status"), rs.getString("response")), idempotencyKey );
switch (record.status) { case "COMPLETED": return new Response(record.response); case "IN_PROGRESS": throw new IllegalStateException("Request already in progress"); case "FAILED": throw new IllegalStateException("Previous attempt failed, safe to retry"); default: throw new IllegalStateException("Unknown barrier state: " + record.status); } } }
private boolean isPostgres() { // Detect DB type from DataSource or JdbcTemplate if needed return true; // placeholder, implement detection }
private String doBusinessLogic(String payload) { return "processed:" + payload; }
private static class IdempotencyRecord { final String status; final String response;
IdempotencyRecord(String status, String response) { this.status = status; this.response = response; } }}
