The Outbox Pattern: Publishing Events Without Losing Them

Writing to your database and telling other services about it are two different operations against two different systems, and most codebases treat them as if they were one. They aren’t, and the gap between them is where events quietly disappear.

The gap, concretely

Say an order service saves a new order, then publishes an OrderCreated event so the shipping and notification services can react.

// The naive version
async function createOrder(orderData) {
  const order = await db.orders.insert(orderData);   // succeeds
  await messageBroker.publish('OrderCreated', order); // crashes here, or broker is down
  return order;
}

If the process crashes, gets killed by a deploy, or the broker connection times out between those two lines, the order exists in your database but nobody downstream ever hears about it. No exception bubbles up to the caller, because the database write already returned success. The order sits there, correctly saved, silently unshippable, until someone notices days later that a customer never got their confirmation email.

You can’t wrap the database write and the broker publish in one transaction, because they’re different systems. Postgres has no concept of a Kafka topic, and Kafka has no concept of a Postgres row. There is no BEGIN that spans both.

The outbox pattern’s actual mechanism

The fix is to stop treating the event as something that needs to reach the broker immediately, and instead treat it as data that needs to reach your own database immediately, in the same transaction as the business write. A separate, independent process is then responsible for getting it from your database to the broker.

CREATE TABLE outbox (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  aggregate_type TEXT NOT NULL,
  aggregate_id TEXT NOT NULL,
  event_type TEXT NOT NULL,
  payload JSONB NOT NULL,
  created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
  processed_at TIMESTAMPTZ
);

// The outbox version
async function createOrder(orderData) {
  return db.transaction(async (tx) => {
    const order = await tx.orders.insert(orderData);
    await tx.outbox.insert({
      aggregate_type: 'Order',
      aggregate_id: order.id,
      event_type: 'OrderCreated',
      payload: order,
    });
    return order; // both rows commit together, or neither does
  });
}

Both inserts are ordinary rows in the same database, in the same local transaction. Postgres (or whatever your database is) already guarantees atomicity within a single transaction, so this part requires no new infrastructure. Either both the order and its outbox row exist, or neither does. There is no window where the order is saved but the event is unrecorded.

The relay: getting rows out of the outbox

The outbox table isn’t the finish line, it’s a durable staging area. Something still has to move those rows to the actual message broker so other services can consume them. Two common approaches:

Polling relay. A background worker queries for unprocessed rows on an interval, publishes each to the broker, and marks it processed.

async function relayOutboxEvents() {
  const rows = await db.outbox.findUnprocessed({ limit: 100 });
  for (const row of rows) {
    await messageBroker.publish(row.event_type, row.payload);
    await db.outbox.markProcessed(row.id);
  }
}

setInterval(relayOutboxEvents, 500); // poll every 500ms

Simple to build and reason about. The cost is latency (events wait up to one polling interval before going out) and repeated read load on the outbox table as it grows, which means you also need a retention job to prune processed rows.

Change data capture (CDC). A tool like Debezium reads the database’s write-ahead log directly and streams new outbox rows to the broker as they’re written, without polling the table at all.

CDC gets you much lower latency and removes the polling overhead entirely, at the cost of running and operating another piece of infrastructure (the CDC connector) and a bit more operational surface area to monitor.

Most teams start with a polling relay because it needs nothing beyond application code, and move to CDC once event latency or database load from polling becomes a real, measured problem, not a hypothetical one.

What the outbox pattern doesn’t give you

Not exactly-once delivery. If the relay publishes an event and then crashes before marking the row processed, it will publish that same event again on restart. The pattern gives you at-least-once delivery, which is a real and useful guarantee, but consumers still need to handle a duplicate arriving. That’s the same idempotency discipline covered in the idempotency key pattern for safe API retries: give each event a stable identifier and have consumers check whether they’ve already processed it before acting.

Not ordering across aggregates. Rows within the outbox for the same aggregate typically publish in insertion order, but if you’re relying on strict global ordering across every event type, that needs to be designed into your relay and broker setup explicitly, it isn’t free.

Not a replacement for the saga pattern. The outbox pattern solves reliably publishing one event from one service. A saga coordinates a multi-step business transaction across several services, often using events published this exact way as its underlying transport. The two patterns compose, they don’t compete.

When to skip it

If an occasionally-lost event is genuinely tolerable, an internal metrics ping, a non-critical analytics event, the naive publish-after-write approach is simpler and the outbox pattern is unnecessary complexity. Reach for it when a missed event has a real consequence: an order that never ships, a payment confirmation another service depends on, an inventory adjustment that silently drifts out of sync. That’s the class of problem the extra table and relay process are actually buying you protection against.