Duplicate events

Problem : 

Duplicate events and messages are being generated during activities such as merging, unmerging, and updating entities.

Cause : 

Amazon SQS retains copies of messages across multiple servers to ensure redundancy and high availability. On rare occasions, one of the servers that holds a copy of a message may be unavailable at the time you receive or delete a message.
 
In such instances, the copy of the message remains on the unavailable server, and you may receive that message copy again when retrieving messages. Please refer to the AWS SQS documentation for more details. 

Solution :

The Reltio platform guarantees “at least one message delivery,” which means that while the platform ensures that every message will be delivered at least once, it also allows for the possibility of duplicate messages being received. This delivery guarantee is designed to maximize reliability, but it requires careful handling on the receiving end to avoid unintended consequences. Because of this potential for receiving the same message multiple times, it is crucial to design your applications to be idempotent (meaning that processing the same message repeatedly will not cause any negative side effects or inconsistent states in your application). In other words, your application logic should be robust enough to handle duplicate messages gracefully, ensuring that repeated processing does not lead to errors, data corruption, or unintended duplication of actions.