Will the correct winner be applied to the relationship table in the DataBricks connector?

Question

I can not find the correct winner for the start or end objects in the relationship DeltaLake table. Do you know if this is expected?

Answer

DPH events are not explicitly created to update the relationship table in the Databricks connector to ensure the correct winner entity ID. However, various events and processes within the Data Pipeline Hub (DPH) and connectors like Databricks impact data synchronization and entity updates. To determine a correct winner/loser for a merge operation, you would query the merge table, not the relationship or the links table. As you can see in the documentation below, the relations table contains the relations' start and end objects.

• If entity A is related to entity B, there will be a record in data bricks for the relationship between A and B.
• Subsequently, if B is merged with some other entity, C, this won’t update the relation in data bricks, as we don’t have any event from the platform.
• So, though B does not exist in the system, the relation between A and B will exist in data bricks, and this needs to be resolved using the merge table.