CTO @ Beautron.AI · AI Product Engineer
2 min read
Architecture / Event-driven / RabbitMQ / E-commerce
The outbox pattern: reliable events for e-commerce
Saving an order and publishing "order placed" are two writes to two systems, and one of them will eventually fail. The transactional outbox makes event-driven commerce reliable without distributed transactions.
The e-commerce platform I architected for the Samsung Nepal online store uses an event-driven design with RabbitMQ: payment, inventory and real-time order tracking all react to events such as "order placed" and "payment confirmed". Event-driven systems are flexible and scale well, but they hide one classic trap: the dual write.
The dual-write problem
When a customer places an order, the service has to do two things:
- save the order in the database, and
- publish an
OrderPlacedevent to the message broker.
These are two different systems, so there's no transaction covering both. Whichever order you do them in, a crash in between leaves you inconsistent:
- save, then crash before publishing: the order exists, but inventory is never reserved and the customer never gets a confirmation;
- publish, then crash before saving: downstream services act on an order that doesn't exist.
Retries don't fix this, because the process that would retry is the one that crashed.
The transactional outbox
The fix is to turn the two writes into one. In the same database transaction that saves the order, insert the event into an outbox table:
BEGIN TRANSACTION;
INSERT INTO orders (...) VALUES (...);
INSERT INTO outbox (id, type, payload, created_at)
VALUES (NEWID(), 'OrderPlaced', @json, SYSUTCDATETIME());
COMMIT;
Either both rows exist or neither does. A separate relay process then reads unsent outbox rows, publishes them to RabbitMQ and marks them as sent. If the relay crashes, it simply publishes the same rows again on restart.
Consumers must be idempotent
The outbox guarantees at-least-once delivery, so consumers will sometimes see the same event twice. Every consumer should record the IDs of events it has processed and skip duplicates, ideally in the same transaction as its own state change: the inbox pattern, the mirror image of the outbox.
With both in place you get effectively-once processing without a distributed transaction.
Ordering and real-time tracking
Order tracking needs events for the same order to be applied in sequence: "shipped" must not overtake "paid". Two practical techniques:
- publish per-order events with the order ID as the routing or partition key, so one consumer handles one order's events in order;
- include a version number in each event, and have consumers ignore events older than the state they already hold.
The customer-facing tracking page then reads from a projection that consumers keep up to date, instead of querying every service on each page load.
Operational details that matter
- Keep the outbox small. Delete or archive published rows on a schedule.
- Alert on age, not just errors. "The oldest unsent outbox row is ten minutes old" is the earliest sign that the relay or broker is unhealthy.
- Version your event payloads. Consumers deploy on their own schedules; add fields, don't rename or remove them.
- Use a dead-letter queue. A malformed event should be parked for inspection, not retried forever.
Takeaways
- Saving state and publishing an event is a dual write; assume it will fail halfway.
- Write the event to an outbox table in the same transaction, and relay it afterwards.
- Make consumers idempotent with an inbox, and use keys or versions to keep per-entity ordering.
- Monitor outbox age, version your events and dead-letter poison messages.
Working on something similar? Let's talk — or ask ARA, the assistant in the corner, about this article.