Why am I not getting any events passed to my Snowflake connector?

Question

We are configuring the Snowflake Data Warehouse connector to publish MDM data into Snowflake through the console. We successfully created the pipeline, but it is not publishing any data to the blob storage even though there are changes in Reltio. 

Answer

  • Validate that streaming is enabled in the physical tenant configuration.

"streamingConfig": {
     "streamingEnabled": true, <<<===Make sure that it is set to true
     "streamingAPIEnabled": true, <<<===Make sure that it is set to true
     "analyzeOvChanges": false,
     "emptyStartEndRelationCrosswalks": false,
     "largeObjectsSupport": false,
     "skipPayload": false,
     "payloadType": "SNAPSHOT",
     "ovOnly": false,

 

{
                 "payloadType": "SNAPSHOT",
                 "type": "queue",
                 "provider": "dph",
                 "name": "<environment>-datapipeline-events_<tenant>",
                 "dtssQueue": false,
                 "groupingQueue": false,
                 "dataPipelineQueue": true,
                 "enabled": true,  <<<<=====  Make sure that it is set to true
                 "format": "KRYO"
             }

 

"adapters": [
            {

                "type": "snowflake",
                "enabled": true,
                "dataFilteringEnabled": false,
                "name": "Snowflake",
                "serializeInitialSourcesInCrosswalks": false,
                "stagingDataRetentionInDays": 14,
                "cloudProvider": "AZURE",
                "azureConfig": {
                    "azureCredentials": {
                        "authMethod": "KEYS",
                        "storageAccountName": "<accountName>"
                  }
                },

                "ovOnly": false,
                "stagingBucket": "<queuebucket name>",
                "attributeFormat": "STANDARD",
                "useLegacyUpdatedTime": false
          }
        ]

Ensure the adapter is valid by using the following API and checking for a status of 200. If not, walk through the connector setup again at https://docs.reltio.com/en/integrate/reltio-data-pipelines/reltio-data-pipeline-for-snowflake for the correct cloud platform with the customer.

POST https://<environment>-data-pipeline-hub.reltio.com/api/tenants/<tenant>/adapters/Snowflake/validate 
  • Check to make sure that events are processing.
    • Check external queue

Run the following API for a specific entity to validate the processed event across the pipeline.




GET https://<environment>-data-pipeline-hub.reltio.com/api/tenants/<tenantid>/monitoring/_entityMonitoring?entityId=<entityID>

 

Was this article helpful?
0 out of 0 found this helpful

Comments

0 comments

Please sign in to leave a comment.