Message Queues vs Pub/Sub vs Event Streaming: Which One Do You Actually Need

Three different tools all claim to solve “get a message from one service to another,” and picking the wrong one doesn’t usually break in testing, it breaks the first time something downstream goes down or you need to fan a message out to more than one place. The distinction that actually matters isn’t throughput or vendor marketing, it’s a much narrower question: does this message need to go to exactly one consumer, to every consumer, or does it need to still exist after everyone’s read it once?

Message queues: exactly one consumer gets each message

A message queue (SQS, RabbitMQ, a Redis-backed queue) is built for distributing work across a pool of workers. Multiple workers can listen on the same queue, but each message is delivered to exactly one of them.

// Producer: enqueue a job
await queue.send('process-image', { imageId: '4821', size: 'thumbnail' });

// Consumer: any one of N workers picks this up, not all of them
queue.consume('process-image', async (job) => {
  await resizeImage(job.imageId, job.size);
  await job.ack(); // remove from queue once processed
});

Add a second worker process listening on process-image, and jobs get split between the two, not duplicated to both. That’s the entire point: a queue is a load-balancing mechanism for work, not a broadcast mechanism for notifications. Once a job is acknowledged, it’s gone from the queue. There’s no replay, no second reader coming along later to see what happened.

This is the right tool when you have a pool of interchangeable workers processing a stream of independent jobs: image resizing, sending emails, generating PDFs. Nobody else needs to know that job happened; it just needs to happen exactly once, by exactly one worker.

Pub/sub: every subscriber gets its own copy

Publish/subscribe (SNS, Google Pub/Sub, a Redis pub/sub channel) inverts that guarantee. Every subscriber to a topic receives every message published to it, independently of every other subscriber.

// Producer: publish an event, doesn't know or care who's listening
await pubsub.publish('order.created', { orderId: '9910', total: 84.50 });

// Three independent subscribers, all three receive every event
pubsub.subscribe('order.created', 'shipping-service', handleShipping);
pubsub.subscribe('order.created', 'analytics-service', handleAnalytics);
pubsub.subscribe('order.created', 'email-service', handleConfirmationEmail);

Add a fourth subscriber tomorrow, and it starts receiving its own copy of every future event without touching the other three. That’s fan-out, and it’s the reason pub/sub fits domain events, “an order was created,” “a user signed up”, where an unknown or growing number of independent parts of the system need to react without being coupled to each other.

The tradeoff: most pub/sub systems don’t retain messages after delivery. If analytics-service is down when an event publishes, that specific event is typically lost for that subscriber once its retry window (if any) expires. Pub/sub answers “who needs to know right now,” not “what happened, and can I look at it again later.”

Event streaming: a durable, replayable log

Event streaming platforms (Kafka, Redpanda, AWS Kinesis) solve a different problem again: an ordered, durable log of events that multiple independent consumer groups can read at their own pace, including replaying from an earlier point.

// Producer: append to the log, partitioned by key for ordering
await kafka.produce('order-events', {
  key: order.id,
  value: { type: 'OrderCreated', orderId: order.id, total: order.total },
});

// Consumer group A processes from wherever it left off
const consumerA = kafka.consumer({ groupId: 'shipping-service' });
await consumerA.subscribe({ topic: 'order-events' });

// Consumer group B, added six months later, can replay from the beginning
const consumerB = kafka.consumer({ groupId: 'fraud-analysis-v2' });
await consumerB.subscribe({ topic: 'order-events', fromBeginning: true });

That fromBeginning: true is the capability neither a queue nor pub/sub gives you by default. Events aren’t deleted when a consumer reads them, they’re retained per the topic’s configured retention period (hours, days, or indefinitely), and each consumer group tracks its own read position independently. If fraud-analysis-v2 gets added as a new service next quarter, it can process the entire history of order events from day one, not just events published after it started listening.

That capability isn’t free. Running a streaming platform well means operating partitioned, ordered logs, managing consumer group offsets, and thinking about retention and storage costs at a different scale than a queue or pub/sub topic requires. It’s also frequently the transport underneath event-driven architectures built on Kafka or Redis Streams, and pairs naturally with the outbox pattern for guaranteeing an event actually gets published once your database write commits.

The decision, side by side

	Message queue	Pub/sub	Event streaming
Delivery	Exactly one consumer per message	Every subscriber, independently	Every consumer group, independently
Message lifetime	Removed after ack	Gone after delivery (usually)	Retained per topic config, replayable
Best fit	Distributing work across a worker pool	Broadcasting domain events to unknown/growing subscribers	Audit trails, replay, multiple consumers reading history at different rates
Adding a new consumer later	Splits existing work, doesn’t duplicate it	Starts receiving new events only	Can replay from the beginning of retained history
Operational complexity	Low to moderate	Low to moderate	Higher: partitioning, offsets, retention

Getting it wrong is a production problem, not a testing one

The failure modes here rarely show up until real conditions hit. A queue used where fan-out was actually needed looks fine until someone adds a second consumer and discovers jobs are being split, not duplicated, meaning half of what was supposed to happen silently isn’t. Pub/sub used where replay was needed looks fine until a downstream service has an outage and comes back up with a permanent gap in what it processed, with no way to recover the missed events short of manually reconstructing them from somewhere else.

Before reaching for infrastructure, answer one question honestly: does this specific piece of work need to happen exactly once by exactly one worker, does it need to notify everyone who currently cares, or does something later need to be able to replay what happened from the start? That answer picks the category. Vendor benchmarks and throughput numbers matter after that, not before it. If you’re scoping this decision as part of a larger architecture review, it’s worth pairing with our technical due diligence framework so the messaging layer gets evaluated alongside the rest of the system, not in isolation.