Integration & APIs/
Lesson

When an integration breaks at 2 AM, your logs are the first thing you reach for. If those logs are unstructured text like "Payment failed for user 123", you are going to have a bad time. Structured loggingWhat is structured logging?Writing log entries as machine-readable JSON objects with consistent fields instead of plain text, making them searchable by log analysis tools. and correlation IDs turn your logs from a wall of text into a searchable, traceable system that actually helps you find the problem.

This lesson focuses specifically on logging for integrations, the patterns you need when your code calls external APIs, receives webhooks, and coordinates across services.

AI pitfall
AI-generated logging code almost always uses console.log with string concatenation. This produces unstructured text that is unsearchable, unfilterable, and useless in production. Always ask AI to use a structured logger (Pino or Winston) with JSON output, and verify it actually does.

Why structured logs matter for integrations

Plain text logs look fine in development:

2024-03-15 10:23:01 INFO: Calling Stripe API to create payment intent
2024-03-15 10:23:02 ERROR: Stripe API returned 402 - Card declined
2024-03-15 10:23:02 INFO: Sending webhook to partner notification URL

But when you have 50 partners, 10 services, and thousands of requests per minute, you cannot grep your way through this. Structured logs fix this by outputting machine-readable JSONWhat is json?A text format for exchanging data between systems. It uses key-value pairs and arrays, and every programming language can read and write it.:

json
{
  "level": "error",
  "msg": "External API call failed",
  "correlationId": "req-abc-123",
  "partner": "stripe",
  "endpoint": "/v1/payment_intents",
  "statusCode": 402,
  "duration": 1203,
  "timestamp": "2024-03-15T10:23:02.451Z"
}

Now you can filter by partner=stripe AND statusCode>=400 in your log aggregator and instantly see every Stripe failure in the last hour.

02

Correlation IDs: the thread that ties everything together

A single user action, say, placing an order, might trigger your APIWhat is api?A set of rules that lets one program talk to another, usually over the internet, by sending requests and getting responses., which calls Stripe for payment, sends a webhookWhat is webhook?An HTTP request that a service sends to your server when a specific event occurs, like a payment completing. Your server processes the event automatically. to a fulfillment partner, and updates an inventory service. That is four services. Without a correlation ID, you have four separate log streams with no way to connect them.

Generating the correlation ID

Generate it once at the entry point. Use a UUIDWhat is uuid?Universally Unique Identifier - a 128-bit random string used as a globally unique ID for database records, events, or request tracking. or a shorter nanoid:

import { randomUUID } from 'crypto';
import pino from 'pino';

const logger = pino({ level: 'info' });

// Middleware: generate or extract correlation ID
app.use((req, res, next) => {
  // Accept from upstream if it already exists
  const correlationId = req.headers['x-correlation-id'] as string
    || randomUUID();

  // Attach to request for downstream use
  req.correlationId = correlationId;

  // Set on response so the caller can see it too
  res.setHeader('x-correlation-id', correlationId);

  // Create a child logger with the ID baked in
  req.log = logger.child({ correlationId });

  next();
});

Every log line from this request now automatically includes the correlationId field without you having to pass it manually.

Propagating across HTTPWhat is http?The protocol browsers and servers use to exchange web pages, API data, and other resources, defining how requests and responses are formatted. calls

When you call an external API, forward the correlation ID:

async function callPartnerAPI(
  req: Request,
  endpoint: string,
  payload: unknown
) {
  const start = Date.now();

  req.log.info({
    msg: 'Calling partner API',
    partner: 'fulfillment-service',
    endpoint,
  });

  const response = await fetch(endpoint, {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      'x-correlation-id': req.correlationId,  // Propagate!
      'Authorization': `Bearer ${process.env.PARTNER_API_KEY}`,
    },
    body: JSON.stringify(payload),
  });

  const duration = Date.now() - start;

  req.log.info({
    msg: 'Partner API responded',
    partner: 'fulfillment-service',
    endpoint,
    statusCode: response.status,
    duration,
  });

  return response;
}

If the fulfillment service also logs with the same correlation ID, you can search for correlationId=req-abc-123 and see the entire request journey across both services.

Good to know
The correlation ID is the single most important field in integration logging. Without it, debugging a distributed problem means manually matching timestamps across 4 different log streams, a process that takes hours instead of seconds. Generate it once at the entry point, propagate it to every downstream call.
03

Setting up Pino for integration logging

Pino is the go-to structured logger for Node.js because of its speed. It serializes JSONWhat is json?A text format for exchanging data between systems. It uses key-value pairs and arrays, and every programming language can read and write it. faster than Winston, which matters when you are logging every external APIWhat is api?A set of rules that lets one program talk to another, usually over the internet, by sending requests and getting responses. call.

import pino from 'pino';

const logger = pino({
  level: process.env.LOG_LEVEL || 'info',
  // Redact sensitive fields automatically
  redact: {
    paths: [
      'req.headers.authorization',
      'req.headers["x-api-key"]',
      'payload.creditCard',
      'payload.ssn',
      'payload.password',
      'response.body.token',
    ],
    censor: '[REDACTED]',
  },
  // Add service name to every log line
  base: {
    service: 'order-service',
    version: process.env.APP_VERSION || 'unknown',
  },
  // Customize serializers for request/response
  serializers: {
    req: pino.stdSerializers.req,
    res: pino.stdSerializers.res,
    err: pino.stdSerializers.err,
  },
});

export default logger;

The redact configuration is critical. Without it, you will inevitably log an API key or a customer's credit card number. Pino replaces those fields with [REDACTED] before the log is written, the sensitive data never hits your log aggregator.

Edge case
AI-generated code frequently logs the full response body from external API calls. This is dangerous because API responses often contain PII (customer emails, addresses) or secrets (tokens in response bodies). Log the shape, status code, duration, body size, key IDs, not the content.
04

What to log vs. what not to log

This is the most common mistake in integration logging: either logging too little (you cannot debug anything) or logging too much (you leak secrets and blow up your storage bill).

Log thisDo not log this
Correlation ID on every lineFull request/response bodies (too large, may contain PII)
Partner name and endpoint URLAPI keys, tokens, or auth headers
HTTP status code and methodCredit card numbers, SSNs, passwords
Request/response duration in msFull customer addresses or phone numbers
Request body size (bytes)Internal database connection strings
Error messages and error codesSession tokens or JWT contents
Retry attempt numberRaw webhook payloads (may contain PII)
Partner-specific request IDHealth check pings (too noisy)

Logging the shape, not the content

For integration calls, log enough to diagnose without exposing sensitive data:

// Good: log the shape
req.log.info({
  msg: 'Stripe API response',
  statusCode: 200,
  duration: 342,
  bodySize: JSON.stringify(responseBody).length,
  paymentIntentId: responseBody.id,
  paymentStatus: responseBody.status,
});

// Bad: log the full body
req.log.info({
  msg: 'Stripe API response',
  body: responseBody,  // Contains card details, customer email, etc.
});
05

Winston alternative setup

If you prefer Winston for its transport flexibility (sending logs to multiple destinations), here is the equivalent setup:

import winston from 'winston';

const logger = winston.createLogger({
  level: process.env.LOG_LEVEL || 'info',
  format: winston.format.combine(
    winston.format.timestamp(),
    winston.format.errors({ stack: true }),
    winston.format.json()
  ),
  defaultMeta: {
    service: 'order-service',
    version: process.env.APP_VERSION || 'unknown',
  },
  transports: [
    new winston.transports.Console(),
    // In production: send to your log aggregator
    // new winston.transports.Http({ host: 'logs.example.com' }),
  ],
});

// Create child logger with correlation ID
function createRequestLogger(correlationId: string) {
  return logger.child({ correlationId });
}

export { logger, createRequestLogger };
06

Logging webhookWhat is webhook?An HTTP request that a service sends to your server when a specific event occurs, like a payment completing. Your server processes the event automatically. receipts

Webhooks deserve special logging attention because they arrive asynchronously and you need to prove you received and processed them:

app.post('/webhooks/stripe', async (req, res) => {
  const webhookId = req.headers['stripe-webhook-id'] as string;
  const correlationId = req.headers['x-correlation-id'] as string
    || randomUUID();

  const log = logger.child({ correlationId, webhookId, partner: 'stripe' });

  log.info({ msg: 'Webhook received', eventType: req.body.type });

  try {
    await processWebhookEvent(req.body, log);
    log.info({ msg: 'Webhook processed successfully' });
    res.status(200).json({ received: true });
  } catch (error) {
    log.error({
      msg: 'Webhook processing failed',
      err: error,
      eventType: req.body.type,
    });
    // Still respond 200 to prevent retries if processing is queued
    res.status(200).json({ received: true, queued: true });
  }
});

The webhook ID from the providerWhat is provider?A wrapper component that makes data available to all components nested inside it without passing props manually. plus your correlation ID gives you two ways to trace the event, from the partner's side and from yours.

07

Quick reference

ConceptPurposeHeader / Field
Correlation IDLink logs across servicesx-correlation-id
Request IDIdentify a single HTTP callx-request-id
Partner request IDTrack on the partner's sideVaries (e.g., stripe-request-id)
RedactionRemove secrets from logsPino redact option
Child loggerAdd context without repeating itlogger.child({ ... })