A message broker can lose an acknowledgement in transit. Consumer processes a message, crashes before acking, broker redelivers. Or the consumer acks but the ack is lost, same result. You can only choose between:
- At-most-once: do not retry. Messages can be lost.
- At-least-once: retry until acked. Messages can arrive twice.
Over an unreliable network, delivering exactly once is not something the transport can promise. What you can have is exactly-once processing: at-least-once delivery combined with a consumer whose effect is idempotent.
The robust way to do it is to record "I have handled message X" in the same transaction as the effect:
BEGIN;
INSERT INTO processed (msg_id) VALUES ($1) ON CONFLICT DO NOTHING;
-- if zero rows were inserted, we saw it before: COMMIT and just ack
UPDATE tallies SET total = total + $2 WHERE id = $3;
COMMIT;
-- only now ack the message
Timeline of a crash:
consumer: BEGIN ... UPDATE ... COMMIT ---x(crash before ack)
broker: ........................ redeliver -------->
consumer: BEGIN INSERT conflicts -> already done -> COMMIT -> ack
Details people miss:
- The dedup record needs a retention policy longer than the broker's maximum redelivery window.
- The id must come from the producer (or be stable per business event), not generated on each attempt.
- Side effects outside the database (an email, an HTTP call) are not covered by the transaction. They need their own idempotency key at the receiver.