Reltio CRUD event ordering: why external queue messages may arrive out of order

Overview

Customers may ask whether two Reltio POST API calls sent one after another can be consumed out of order from an external queue. The answer is yes, especially when requests are executed at approximately the same time or processed in parallel.

Reltio message streaming is not FIFO. Reltio processes multiple requests in parallel across multiple nodes, and It does not guarantee that events are processed in the same order as requests are added to the queue. Reltio recommends using objectVersion and message timestamp information to order events instead of relying on queue arrival order.

Reltio event streaming/message streaming for CRUD and Match events delivered to external queues or topics, including destinations such as Amazon SQS/SNS, Google Pub/Sub, Azure Service Bus, and Confluent Kafka.

Symptoms

A downstream consumer may observe:

• Events for the same object arriving out of order
• Events arriving later than the corresponding update time
• Duplicate messages
• Missing objectVersion in some payloads or headers

Reltio Support has an existing article for SQS-specific behavior covering delays, ordering, duplicates, and missing objectVersion.

Cause

Reltio publishes CRUD events asynchronously. Event generation, processing, and delivery are decoupled, so delivery timing is not guaranteed to match transaction timing. In addition, Reltio does not guarantee FIFO ordering for event delivery.

For simultaneous or near-simultaneous updates, a later request may complete processing before an earlier request and enter the event queue first. This means the order in which messages are received from the external queue should not be treated as the authoritative sequence.

Recommended resolution

Downstream systems should determine the correct event sequence using Reltio event metadata, not external queue receive order.

Use objectVersion to order events for the same object. Reltio documentation states that objectVersion is an incremental value updated after every change to the business object, and the message with the highest objectVersion contains the latest object state.

Do not use createdTime, updatedTime, crosswalk timestamps, or queue arrival order as the primary sequencing mechanism. Reltio documents that createdTime and updatedTime indicate the start of an operation, not the time the object record is committed to the database.

When commitTime is available in the event body, it may provide useful event metadata. Reltio Support notes that, for SQS messages, objectVersion is defined in the attribute section and commitTime is defined in the body of the event message.

Downstream processing best practices

Implement idempotent processing so receiving the same message more than once does not create duplicate downstream changes. Reltio Support recommends using identifiers such as eventId or entity identifiers for deduplication.

Track the last processed objectVersion per object URI. Apply an incoming event only when its objectVersion is newer than the last successfully processed version for that object.

If objectVersion is missing, validate the streaming configuration. Reltio CRUD event documentation states that messages contain the objectVersion header only if it is included in JMSEventsFilteringFields based on message streaming configuration.