What idempotency actually means

An idempotentWhat is idempotent?An operation that produces the same result whether you perform it once or multiple times, making retries safe. Ask AI for more operation produces the same result no matter how many times you execute it. In math, multiplying by 1 is idempotent: 5 x 1 x 1 x 1 = 5. In HTTPWhat is http?The protocol browsers and servers use to exchange web pages, API data, and other resources, defining how requests and responses are formatted. Ask AI for more, GET and PUT are supposed to be idempotent. POST is not, and webhooks are POST requests.

Your job is to make your POST webhookWhat is webhook?An HTTP request that a service sends to your server when a specific event occurs, like a payment completing. Your server processes the event automatically. Ask AI for more handler behave idempotently even though the HTTP method is not.

The pattern is clear: absolute operations are naturally idempotent. Relative operations (increment, decrement, append) are not. When AI generates webhook handlers, it almost always uses relative operations for things like inventory and balances, and that is where duplicates cause real damage.

Why duplicates happen

Understanding why duplicates occur helps you design better defenses. It is not just "the network is unreliable", there are specific, predictable scenarios.

The most common cause by far is the timeout retry. Your handler does too much work before responding, the sender times out, and now you have two copies of the same event being processed simultaneously. This is why "respond first, process later" is not just a performance optimization, it is a correctness requirement.

Implementation with Redis

Redis is the most popular choice for idempotency tracking because it is fast (sub-millisecond reads) and supports TTLWhat is ttl?Time-to-Live - a countdown attached to cached data that automatically expires it after a set number of seconds. Ask AI for more natively. Here is a production-quality implementation:

const redis = require('redis');
const client = redis.createClient();

async function processWebhook(event) {
  const eventId = event.id;
  const lockKey = `webhook:lock:${eventId}`;
  const processedKey = `webhook:processed:${eventId}`;

  // 1. Check if already processed (idempotency)
  const alreadyProcessed = await client.get(processedKey);
  if (alreadyProcessed) {
    console.log(`Duplicate webhook: ${eventId}`);
    return JSON.parse(alreadyProcessed); // Return same response
  }

  // 2. Distributed lock (avoid race condition between duplicates)
  const locked = await client.set(lockKey, '1', 'EX', 60, 'NX');
  if (!locked) {
    // Another instance is processing this same event right now
    await new Promise(resolve => setTimeout(resolve, 1000));
    return processWebhook(event); // Retry (will hit the "already processed" check)
  }

  try {
    // 3. Execute business logic
    const result = await handleEvent(event);

    // 4. Store result + mark as processed (TTL 7 days)
    await client.setex(
      processedKey,
      7 * 24 * 60 * 60,
      JSON.stringify(result)
    );

    return result;
  } finally {
    // 5. Always release the lock, even on error
    await client.del(lockKey);
  }
}

There are three layers of protection here: the idempotency check (step 1), the distributed lock (step 2), and the processed marker (step 4). Each layer handles a different failure mode.

Implementation with a database

If you do not want to add Redis to your stack, a database table works just as well. The advantage is that your idempotency state is durable and transactional, it survives restarts and can be wrapped in the same transactionWhat is transaction?A group of database operations that either all succeed together or all fail together, preventing partial updates. Ask AI for more as your business logic.

-- The processed_events table
CREATE TABLE processed_events (
  event_id VARCHAR(255) PRIMARY KEY,
  result JSONB,
  processed_at TIMESTAMP DEFAULT NOW(),
  expires_at TIMESTAMP DEFAULT NOW() + INTERVAL '7 days'
);

-- Index for cleanup job
CREATE INDEX idx_expires_at ON processed_events (expires_at);

Pattern: transactionWhat is transaction?A group of database operations that either all succeed together or all fail together, preventing partial updates. Ask AI for more with deduplication

This is the gold standard for webhookWhat is webhook?An HTTP request that a service sends to your server when a specific event occurs, like a payment completing. Your server processes the event automatically. Ask AI for more idempotency. By putting the deduplication check and the business logic in the same database transaction, you get atomic exactly-once processing:

async function handlePaymentWebhook(event) {
  const paymentId = event.data.object.id;

  return await db.transaction(async (trx) => {
    // 1. Try to insert into processed_events
    const [processed] = await trx('processed_events')
      .insert({
        event_id: event.id,
        processed_at: new Date()
      })
      .onConflict('event_id')
      .ignore()
      .returning('*');

    // 2. If insert was ignored, this event was already handled
    if (!processed) {
      console.log(`Event ${event.id} already processed`);
      return { status: 'already_processed' };
    }

    // 3. Idempotent business logic (UPSERT, not INSERT)
    const [order] = await trx('orders')
      .insert({
        payment_id: paymentId,
        status: 'paid',
        amount: event.data.object.amount
      })
      .onConflict('payment_id')
      .merge()  // UPSERT: update if exists, insert if not
      .returning('*');

    return { status: 'success', orderId: order.id };
  });
}

Notice the onConflict('payment_id').merge() on the orders table. Even inside the idempotency check, the business logic itself is also idempotentWhat is idempotent?An operation that produces the same result whether you perform it once or multiple times, making retries safe. Ask AI for more. This is defense in depth, if the idempotency check somehow fails (table gets truncated, TTLWhat is ttl?Time-to-Live - a countdown attached to cached data that automatically expires it after a set number of seconds. Ask AI for more expires), the business logic still does the right thing.

Handling non-idempotentWhat is idempotent?An operation that produces the same result whether you perform it once or multiple times, making retries safe. Ask AI for more side effects

Some operations are inherently non-idempotent: sending an email, charging a credit card, calling an external APIWhat is api?A set of rules that lets one program talk to another, usually over the internet, by sending requests and getting responses. Ask AI for more. You cannot "undo" a sent email. The strategy here is to gate these operations behind the idempotency check:

async function processOrder(event) {
  // Step 1: Idempotent database operations (safe to retry)
  const order = await upsertOrder(event);

  // Step 2: Check if side effects already executed
  if (order.emailSent) {
    return; // Already sent, do not send again
  }

  // Step 3: Execute side effect
  await sendConfirmationEmail(order.customerEmail);

  // Step 4: Mark side effect as done (in same transaction if possible)
  await db('orders')
    .where({ id: order.id })
    .update({ emailSent: true });
}

This pattern, flag-gated side effects, ensures that each non-idempotent action runs exactly once, even if the overall handler runs multiple times.

Error handling and recovery

When processing fails, you need to decide: should the sender retry, or should you handle recovery yourself?

app.post('/webhooks/stripe', async (req, res) => {
  try {
    const event = verifySignature(req);

    // Acknowledge quickly
    res.status(200).send('Received');

    // Process with internal retry logic
    await withRetry(
      () => processWebhook(event),
      {
        retries: 3,
        backoff: 'exponential',
        onRetry: (error, attempt) => {
          console.log(`Retry ${attempt} for ${event.id}: ${error.message}`);
        }
      }
    );
  } catch (error) {
    if (!res.headersSent) {
      res.status(500).send('Error');
    }

    // After all retries exhausted, alert for manual review
    console.error('Webhook processing failed permanently:', error);
    await alertOpsTeam({
      eventId: event?.id,
      error: error.message,
      payload: req.body
    });
  }
});