System Design/
Lesson

The producer/consumer pattern is the backbone of asynchronous systems. A producer creates messages and pushes them into a queue. A consumer pulls messages out and processes them. Neither knows about the other. This separation lets you scale each side independently, handle failures gracefully, and smooth out traffic spikes.

The basic pattern

// Producer: creates work items
class OrderProducer {
  constructor(queue) {
    this.queue = queue;
  }

  async submitOrder(order) {
    // Validate and save order first (synchronous, user-facing)
    const savedOrder = await db.orders.create(order);

    // Then publish for async processing
    await this.queue.publish('orders.new', {
      orderId: savedOrder.id,
      items: savedOrder.items,
      userId: savedOrder.userId,
      timestamp: Date.now()
    });

    return savedOrder; // User gets immediate response
  }
}

// Consumer: processes work items
class OrderConsumer {
  async start() {
    await queue.subscribe('orders.new', async (message) => {
      try {
        const { orderId, items } = message.data;
        await this.reserveInventory(orderId, items);
        await this.calculateShipping(orderId);
        await this.sendConfirmationEmail(orderId);
        message.ack(); // Tell queue: processing succeeded
      } catch (err) {
        console.error(`Failed to process order ${message.data.orderId}:`, err);
        message.nack(); // Tell queue: processing failed, redeliver
      }
    });
  }
}

The producer does not care if the consumer is fast, slow, or temporarily down. It drops the message and moves on. The consumer processes at its own pace. If messages pile up, you add more consumers.

02

Competing consumers (load balancing)

When one consumer cannot keep up with the message rate, you add more. Multiple consumers pull from the same queue, and each message goes to exactly one of them. This is called the competing consumer pattern because the consumers "compete" for the next message.

// Three competing consumers on the same queue
// Each message is delivered to exactly ONE consumer

// Consumer 1
queue.subscribe('emails.send', async (msg) => {
  await sendEmail(msg.data);
  msg.ack();
});

// Consumer 2 (same queue, same logic, different process)
queue.subscribe('emails.send', async (msg) => {
  await sendEmail(msg.data);
  msg.ack();
});

// Consumer 3
queue.subscribe('emails.send', async (msg) => {
  await sendEmail(msg.data);
  msg.ack();
});

// Queue distributes messages round-robin:
// Message 1 -> Consumer 1
// Message 2 -> Consumer 2
// Message 3 -> Consumer 3
// Message 4 -> Consumer 1
// ...

This is horizontal scalingWhat is horizontal scaling?Adding more machines to handle increased load, rather than upgrading a single machine to be more powerful. for message processing. Need more throughputWhat is throughput?The number of requests or operations a system can handle per unit of time, like requests per second.? Add more consumers. A consumer crashes? The others pick up the slack, and the dead consumer's unacknowledged messages get redelivered.

03

Fan-out (pub/subWhat is pub/sub?A messaging pattern where senders publish events to a channel and any number of listeners receive them in real time.)

Sometimes you want every subscriber to receive every message. When an order is placed, the inventory service needs to know, the analytics service needs to know, and the notification service needs to know. Each of them needs its own copy.

// Fan-out: every subscriber gets every message
// RabbitMQ: use a fanout exchange
// Kafka: use different consumer groups
// SQS: use SNS topic -> multiple SQS queues

// Publisher
await exchange.publish('order.placed', orderData);

// Subscriber 1: Inventory service (gets ALL order.placed messages)
exchange.subscribe('order.placed', 'inventory-queue', async (msg) => {
  await reserveStock(msg.data.items);
  msg.ack();
});

// Subscriber 2: Analytics service (also gets ALL order.placed messages)
exchange.subscribe('order.placed', 'analytics-queue', async (msg) => {
  await trackOrderMetrics(msg.data);
  msg.ack();
});

// Subscriber 3: Notification service (also gets ALL messages)
exchange.subscribe('order.placed', 'notification-queue', async (msg) => {
  await sendOrderNotification(msg.data.userId);
  msg.ack();
});

The key difference: in competing consumers, adding a consumer splits the work. In fan-out, adding a subscriber adds a new copy of all messages.

AspectCompeting consumersFan-out
Message deliveryEach message to ONE consumerEach message to ALL subscribers
PurposeLoad balancing, parallel processingBroadcasting, event notification
Add a consumerWork splits across more workersNew subscriber gets its own copy
Use caseEmail sending, image processingOrder events, audit logging
Queue modelSingle queue, multiple readersExchange/topic with multiple queues
Failure impactOther consumers pick up slackOnly that subscriber's queue backs up
04

Consumer groups (Kafka concept)

Kafka introduced consumer groups, which elegantly combine both patterns. A consumer group is a named set of consumers that together process all messages from a topic. Within a group, messages are distributed (competing). Across groups, messages are duplicated (fan-out).

// Kafka consumer groups: best of both worlds

// Group "inventory-service" (3 instances for load balancing)
// Each order goes to exactly ONE instance in this group
const inventoryConsumer = new KafkaConsumer({
  groupId: 'inventory-service',
  topics: ['orders']
});

// Group "analytics-service" (2 instances)
// Each order ALSO goes to exactly ONE instance in this group
const analyticsConsumer = new KafkaConsumer({
  groupId: 'analytics-service',
  topics: ['orders']
});

// Result:
// Order #1 -> inventory-service (instance 2) AND analytics-service (instance 1)
// Order #2 -> inventory-service (instance 1) AND analytics-service (instance 2)
// Order #3 -> inventory-service (instance 3) AND analytics-service (instance 1)

Each group sees every message (fan-out). Within each group, messages are distributed among instances (competing). This is how you scale individual services independently while ensuring every service gets the data it needs.

05

Message acknowledgment

Acknowledgment (ack) is how consumers tell the queue that processing is done. Without it, the queue has no idea whether the consumer actually handled the message or crashed mid-way.

// Manual acknowledgment (most control)
queue.subscribe('tasks', async (message) => {
  try {
    const result = await processTask(message.data);
    await saveResult(result);
    message.ack(); // "I'm done, remove this message from the queue"
  } catch (err) {
    if (isRetryable(err)) {
      message.nack(); // "I failed, put it back for another attempt"
    } else {
      message.reject(); // "I failed and retrying won't help, send to DLQ"
    }
  }
});

Three acknowledgment actions:

ActionMeaningQueue behavior
ack()Processing succeededRemove message permanently
nack()Processing failed, retryRedeliver to same or different consumer
reject()Processing failed permanentlySend to dead letter queue (DLQ)
06

Visibility timeout

When a consumer picks up a message, the queue hides it from other consumers for a set period. This is the visibility timeout. If the consumer does not acknowledge before the timeout expires, the queue assumes the consumer died and makes the message visible again.

// SQS-style visibility timeout
const consumer = new SQSConsumer({
  queueUrl: 'https://sqs.us-east-1.amazonaws.com/123/orders',
  visibilityTimeout: 30, // seconds
  handleMessage: async (message) => {
    // You have 30 seconds to process and ack

    // For long-running tasks, extend the timeout
    if (willTakeLong(message)) {
      await sqs.changeMessageVisibility({
        ReceiptHandle: message.ReceiptHandle,
        VisibilityTimeout: 120 // extend to 2 minutes
      });
    }

    await processOrder(message.Body);
    // Auto-ack on success (SQS consumer library handles this)
  }
});

Getting the visibility timeout right matters:

TimeoutToo shortToo long
EffectMessage redelivered while still being processedIf consumer dies, message stuck invisible for a long time
ResultDuplicate processingDelayed retry, messages appear "stuck"
FixSet timeout to 2-3x expected processing timeUse heartbeat/extension for long tasks

A common pattern is to set a moderate default (30-60 seconds) and have the consumer extend the timeout for messages that need more time. This gives you fast redelivery for crashes while supporting long-running tasks.

07

Scaling considerations

The producer/consumer pattern gives you independent scaling knobs. Here is how to think about it:

// Monitoring and scaling decisions
async function evaluateScaling(queueMetrics) {
  const {
    messageRate,       // Messages published per second
    processingRate,    // Messages consumed per second
    queueDepth,        // Current messages waiting
    avgProcessingTime, // Seconds per message
    consumerCount      // Current active consumers
  } = queueMetrics;

  // Queue growing? Need more consumers
  if (messageRate > processingRate) {
    const deficit = messageRate - processingRate;
    const consumersNeeded = Math.ceil(deficit * avgProcessingTime);
    console.log(`Need ${consumersNeeded} more consumers to keep up`);
  }

  // Queue too deep? Might need to scale producers down or consumers up
  if (queueDepth > 10000) {
    console.warn(`Queue depth at ${queueDepth}, consider scaling consumers`);
  }

  // Over-provisioned? Scale down to save costs
  if (queueDepth === 0 && processingRate > messageRate * 2) {
    console.log('Consumers idle, consider scaling down');
  }
}

The beauty of this pattern is that producers and consumers scale independently. During a sale event, you might have the same number of web servers (producers) but triple the order-processing consumers. During off-hours, you scale consumers down to save money. The queue absorbs the difference.

AI pitfall
AI-generated consumer code rarely handles partial failures correctly. If your consumer processes a batch of 10 messages and fails on message 7, AI code will often retry the entire batch, re-processing messages 1-6 that already succeeded. Design your consumers to be idempotent so reprocessing is safe, or process messages individually.
Good to know
Queue depth is one of the most useful metrics in your entire system. A growing queue means consumers are falling behind. A consistently empty queue means you are over-provisioned. Set alerts for both, one saves you from outages, the other saves you money.
Edge case
Visibility timeout in SQS (and similar concepts in other queues) is a subtle failure source. If your consumer takes longer than the visibility timeout to process a message, the queue assumes the consumer died and re-delivers the message to another consumer. Now two consumers are processing the same message simultaneously. Always set visibility timeout higher than your worst-case processing time.