Why are we seeing a match event in the Snowflake connector when we have not had any activity?

Question

We are seeing that snowflake connector sending data even if reltio data is not changing. We refreshed the data on 15th after that we have not loaded anything into reltio. I am attaching the S3 file which we got today. The relevant entity ids are '1EHGDbd', '2B8ShUQ'

Result in Snowflake

[
{
"deleted": false,
"potential_matches": [
{
"matchScore": 0,
"matchRules": [
"configuration/entityTypes/Organization/matchGroups/Rule1_1",
"configuration/entityTypes/Organization/matchGroups/Rule3"
],
"entityId": "1EHGDbd"
}
],
"manual_matches": [],
"entityId": "2B8ShUQ",
"not_matches": [],
"version": 2000000000,
"timestamp": 1698080394342,
"insertedTime": 1698080397771,
"objectType": "match"
},
{
"deleted": false,
"potential_matches": [
{
"matchScore": 0,
"matchRules": [
"configuration/entityTypes/Organization/matchGroups/Rule1_1",
"configuration/entityTypes/Organization/matchGroups/Rule3"
],
"entityId": "2B8ShUQ"
}
],
"manual_matches": [],
"entityId": "1EHGDbd",
"not_matches": [],
"version": 2000000000,
"timestamp": 1698080394342,
"insertedTime": 1698080397771,
"objectType": "match"
}
]

Answer

We can see that you did have activity on the two entities that you described.  Here we see that 1EHGDbd had the following event that was processed.
 

 {
     "type": {
         "name": "POTENTIAL_MATCHES_REMOVED"
     },
     "timestamp": 1698189101285,
     "updatedBy": "IncrementalMatchThread",
     "entity_uri": "entities/1EHGDbd",
     "entity_type": "Organization",
     "label": "MAESTRO SWISS CHOCOLATE SDN. BHD.",
     "affectedEntityURIs": [
         "entities/2B8ShUQ"
     ]
},

To store matches data type correctly we need to listen all matches events.  If potential matches are removed we need to remove it from snowflake as well, for this we are listening  all potential matches events

timestamp eventId eventType1 queueAction
2023-10-24 20:51:21.717000 UTC 1698179932126_DRPNa1m M1_ENTITY_CHANGED CREATED
2023-10-24 20:51:21.746000 UTC 1698179932126_DRPNa1m M1_ENTITY_CHANGED SENT
2023-10-24 22:25:48.219000 UTC 1698185149944_HBB5opL ENTITY_CHANGED CREATED
2023-10-24 22:25:48.605000 UTC 1698185149944_HBB5opL ENTITY_CHANGED SENT
2023-10-24 23:11:41.091000 UTC 1698179932126_DRPNa1m M1_ENTITY_CHANGED RECEIVED
2023-10-24 23:11:45.995000 UTC 1698189101285_WCnlb4i ENTITIES_MATCHES_CHANGED CREATED
2023-10-24 23:11:47.293000 UTC 1698189101285_WCnlb4i ENTITIES_MATCHES_CHANGED SENT
2023-10-24 23:11:47.397000 UTC 1698179932126_DRPNa1m M1_ENTITY_CHANGED PROCESSED
2023-10-24 23:11:51.591000 UTC 1698189101285_WCnlb4i ENTITIES_MATCHES_CHANGED RECEIVED
2023-10-24 23:11:52.478000 UTC 1698189101285_WCnlb4i ENTITIES_MATCHES_CHANGED RECEIVED
2023-10-24 23:11:52.843000 UTC 1698189101285_WCnlb4i ENTITIES_MATCHES_CHANGED DATAPIPELINE_PROCESSED
2023-10-24 23:11:53.772000 UTC 1698189101285_WCnlb4i ENTITIES_MATCHES_CHANGED PROCESSED
2023-10-24 23:39:24.629000 UTC 1698185149944_HBB5opL ENTITY_CHANGED RECEIVED
2023-10-24 23:39:27.323000 UTC 1698185149944_HBB5opL ENTITY_CHANGED PROCESSED

 

Was this article helpful?
0 out of 0 found this helpful

Comments

0 comments

Please sign in to leave a comment.