Exactly-Once vs At-Least-Once Delivery: What Your Message Queue Actually Guarantees

Most systems that advertise “exactly-once delivery” are not giving you exactly-once delivery. They’re giving you at-least-once delivery and a deduplication mechanism, which is a genuinely useful and very different thing to design around. Knowing the difference is the entire ballgame for whether your consumer double-charges a customer the day your network has a bad afternoon.

The three delivery guarantees, precisely

At-most-once: a message is delivered zero or one times. It’s never duplicated, but it can be silently lost, typically because the broker or producer gives up after a failed delivery attempt rather than retrying indefinitely. Fire-and-forget publishing with no acknowledgment falls into this category by default.

At-least-once: a message is delivered one or more times. It’s never silently dropped, but duplicates happen under normal failure conditions, a consumer that crashes after finishing the work but before sending its acknowledgment back to the broker will see that same message redelivered when it restarts, because as far as the broker knows, the work never got confirmed.

Exactly-once: a message is delivered precisely once, always. No loss, no duplication, under any failure condition. This is the one that doesn’t actually exist in the general case across an unreliable network.

Why exactly-once is provably impossible, not just hard

This traces back to the two generals problem, a classic result in distributed systems theory. Two generals need to coordinate an attack, communicating only by messenger through hostile territory where any messenger might be captured. General A sends “attack at dawn.” To be confident the message arrived, A wants an acknowledgment back. But that acknowledgment can also be lost, and now B doesn’t know whether A received the ack, so B might send another ack to be safe, which A also can’t be sure arrived. There is no finite number of messages that gives both generals certainty; the coordination problem has no solution over an unreliable channel.

Delivery guarantees are the same problem in different clothes. For a producer to know with certainty that a consumer received and processed a message exactly once, it needs a confirmation, and that confirmation can itself be lost. The producer then can’t tell “the message never arrived” from “it arrived and did its job, but the confirmation didn’t make it back,” and the only safe response to that uncertainty is to resend, which is exactly what reintroduces duplicates. This isn’t a gap current message broker technology hasn’t closed yet. It’s a structural property of any two systems that coordinate over a network where messages can be delayed or dropped.

What “exactly-once” actually ships as

Kafka is the most concrete example, because it’s specific and well documented about what its exactly-once semantics actually cover. The idempotent producer assigns each message a sequence number per producer session, so if a network retry causes the same message to be sent twice, the broker recognizes the duplicate sequence number and writes it only once. The transactional API extends this across a full read-process-write chain, consume from one topic, transform, produce to another, so the whole chain either commits atomically or doesn’t happen at all, no partial results visible to downstream consumers.

That’s a real and useful guarantee, but notice its boundary: it holds for operations that stay entirely inside Kafka. The moment your consumer’s side effect reaches outside that boundary, an HTTP call to a payment provider, a write to a Postgres database, an email send, the transactional guarantee doesn’t extend to it. You’re back to at-least-once delivery for that external effect, and the deduplication burden is yours to handle.

def handle_order_message(message):
    # this check is what turns at-least-once delivery
    # into an effectively-once side effect
    if already_processed(message.id):
        ack(message)
        return

    charge_customer(message.order)   # the actual side effect
    mark_processed(message.id)       # record it atomically with the effect
    ack(message)

CREATE TABLE processed_messages (
    message_id  TEXT PRIMARY KEY,
    processed_at TIMESTAMPTZ NOT NULL DEFAULT now()
);

The unique constraint on message_id does the actual work here, the same mechanism behind idempotency keys on API endpoints: the first attempt to record a given message ID succeeds, any redelivery hits the constraint and is recognized as a duplicate rather than reprocessed. This is what “effectively-once” processing looks like in practice, built on top of an at-least-once delivery guarantee, not a broker-level exactly-once feature that magically covers your database write and your Kafka consume in the same atomic unit.

Designing for what you actually have

Start from the assumption that your queue gives you at-least-once delivery, because for anything with a side effect outside the broker itself, that’s the honest guarantee regardless of what the vendor’s marketing page says. From there, the design question is whether the consumer’s operation is naturally idempotent, setting a field to a fixed value is naturally safe to repeat, incrementing a counter is not, and where it isn’t naturally idempotent, build the deduplication ledger explicitly rather than assuming a platform feature covers it end to end.

Guarantee	Loss risk	Duplicate risk	Typical fit
At-most-once	Yes	No	Metrics/telemetry where an occasional dropped point is fine
At-least-once	No	Yes	Default for most brokers; needs consumer-side dedup for side effects
”Exactly-once” (vendor label)	No, within the broker’s boundary	No, within the broker’s boundary	Chains that stay entirely inside the broker; falls back to at-least-once the moment a side effect leaves it

The failure mode worth actually worrying about isn’t picking the wrong guarantee. It’s trusting a vendor’s exactly-once label to cover a side effect it was never designed to cover, and finding out during an incident review that a customer got charged twice because the payment call happened outside the boundary the guarantee actually protects.