Problem
A consumer that updates its database and then acknowledges the broker can crash between the two steps. The message is redelivered and applied twice, or acknowledged first and lost.
Also known as: inbox pattern, idempotent receiver table
Record each incoming message in the local database within the same transaction as its effects, so redelivery is detected and processing is atomic.
A consumer that updates its database and then acknowledges the broker can crash between the two steps. The message is redelivered and applied twice, or acknowledged first and lost.
The inbox is the receiving counterpart of the outbox. Where the outbox makes publishing atomic with a state change, the inbox makes consuming atomic with its effects.
The unique message id is the heart of it: the database becomes the arbiter of whether a message was already handled.
Two variants exist. In the inline variant, the handler does the work inside the same transaction that records the id, which is simplest. In the deferred variant, the receiver only stores the message and a separate processor handles pending rows, which isolates slow handlers from the broker connection and gives you a queryable backlog.
Within one transaction the handler inserts into inbox, applies business changes and commits. Only then does it acknowledge the broker.
BEGIN;
INSERT INTO inbox(message_id, received_at) VALUES (:id, now())
ON CONFLICT DO NOTHING;
-- if 0 rows inserted: duplicate, skip to ack
UPDATE stock SET qty = qty - :n WHERE sku = :sku;
COMMIT;
broker.ack(id)
The inbox also helps with diagnostics, because each row records when a message arrived and its outcome. Add a status column and attempt count to support retries and manual inspection. Index the id column, keep rows small, and partition the table by time so old data can be dropped cheaply instead of running slow deletes.
Low - table in existing database
Medium - purge jobs and monitoring
Medium - transaction boundaries
Inbox rows never purged and table bloats
Ack sent before commit loses messages
Id chosen by consumer instead of producer breaks dedup