Message Delivery Guarantees: At-Most-Once, At-Least-Once, & Exactly-Once

In distributed messaging systems (such as Apache Kafka, RabbitMQ, AWS SQS, and Google Cloud Pub/Sub), message delivery between producers, brokers, and consumers is categorized into three mathematical guarantee models:

1. At-Most-Once (Zero Duplicates, Possible Data Loss)

• Mechanism: Messages are dispatched with zero acknowledgments (acks=0). On the consumer side, the consumer commits its read offset before processing the message payload.
• Failure Scenario: If the consumer crashes while processing, the offset is already committed; upon restart, the message is skipped.
• Trade-off: Lowest latency, lowest resource overhead, zero duplicates, but messages can be permanently lost.
• Use Case: High-volume telemetry, video streaming metrics, mouse-tracking analytics.

2. At-Least-Once (Zero Data Loss, Possible Duplicates — The Industry Standard)

• Mechanism: The producer retries until it receives a durable broker acknowledgment (acks=all). The consumer processes the message payload and commits its read offset only after processing succeeds.
• Failure Scenario: If the consumer crashes after processing the business logic but before committing the offset to Kafka, the broker will redeliver the message to another consumer instance, resulting in a duplicate message.
• Trade-off: Guarantees zero data loss, but requires downstream consumers to be idempotent.

3. Exactly-Once Processing (Zero Loss, Zero Duplicates)

• Mechanism: Guarantees that every message is processed and its side effects applied exactly once, even in the presence of producer retries, broker failovers, and consumer restarts.