Question
I've tried several times to get new data into our Snowflake schema but with no luck. I've tried re-creating the pipelines, and re-exporting jobs with both incremental and batch options but no new data has made it into the warehouse.
-
Database: y57ngkcXtTgrvLi_DB
-
DataWarehouse: y57ngkcXtTgrvLi_WH
-
DatabaseSchema: y57ngkcXtTgrvLi_SCHEMA
-
Reltio Pipeline ID's: (Recreated several times in attempts to get it to work)
-
y57ngkcXtTgrvLi_1
-
y57ngkcXtTgrvLi_2
-
y57ngkcXtTgrvLi_3
-
-
Query to test with:
-
select * from "y57ngkcXtTgrvLi_DB"."y57ngkcXtTgrvLi_SCHEMA"."entity_Contact_ov";
-
Only shows 8 entities, when there should be 50k
-
-
Pipeline/Job Details:
Pipeline Definition
{
"pipelineId": "y57ngkcXtTgrvLi_poc",
"description": "Snowflake POC Reltio",
"entityTypes": [
{
"type": "CostCenter",
"attributesToIgnore": [],
"analyticsAttributesToIgnore": []
},
{
"type": "Organization",
"attributesToIgnore": [],
"analyticsAttributesToIgnore": []
},
{
"type": "Country",
"attributesToIgnore": [],
"analyticsAttributesToIgnore": []
},
{
"type": "Employee",
"attributesToIgnore": [],
"analyticsAttributesToIgnore": []
},
{
"type": "Location",
"attributesToIgnore": [],
"analyticsAttributesToIgnore": []
},
{
"type": "GeneralLedgerAccount",
"attributesToIgnore": [],
"analyticsAttributesToIgnore": []
},
{
"type": "BusniessUnit",
"attributesToIgnore": [],
"analyticsAttributesToIgnore": []
},
{
"type": "Contact",
"attributesToIgnore": [],
"analyticsAttributesToIgnore": []
}
],
"relationTypes": [
{
"type": "RelatedProducts",
"attributesToIgnore": [],
"analyticsAttributesToIgnore": []
},
{
"type": "affiliatedwith_sfdc",
"attributesToIgnore": [],
"analyticsAttributesToIgnore": []
}
],
"interactionTypes": [],
"additionalTypes": [],
"multiValueForSimpleAttribute": *false*,
"accountDetailsAlias": "snowflake_SalesforcePOC",
"database": "y57ngkcXtTgrvLi_DB",
"dataWarehouse": "y57ngkcXtTgrvLi_WH",
"databaseSchema": "y57ngkcXtTgrvLi_SCHEMA",
"ovOnly": *false*,
"overrideExistingTables": *true*
}
-
Incremental Job: GET https://gus-sales-analytics-connector.reltio.com/api/v1/tenants/y57ngkcXtTgrvLi/jobs/24bdd0a3-5d6d-4a67-b14d-c2fa0727c15d
- Failed Batch Job: GET https://gus-sales-analytics-connector.reltio.com/api/v1/tenants/y57ngkcXtTgrvLi/jobs/d76e944f-05f6-464e-8dbe-a13339ff322f
{
"jobId": "b42539a5-e995-490d-b1b8-50c15dfd52b6",
"type": "INCREMENTAL_BATCH",
"tenantId": "y57ngkcXtTgrvLi",
"pipelineId": "y57ngkcXtTgrvLi_poc",
"status": "FAILED",
"failureReason": "Task failed due to unexpected snowflake run time error, errorCode: SNOWFLAKE_RUNTIME_ERROR, errorMessage: SQL compilation error:\nTable '\"y57ngkcXtTgrvLi_DB\".\"y57ngkcXtTgrvLi_SCHEMA\".\"entity_CostCenter_source\"' does not exist or not authorized., errorDescription: An error occurred while running Snowflake ingestion task. SQL compilation error:\nTable '\"y57ngkcXtTgrvLi_DB\".\"y57ngkcXtTgrvLi_SCHEMA\".\"entity_CostCenter_source\"' does not exist or not authorized.",
"type": "SNOWFLAKE_INGEST_TASK"
Answer
The job failed because there was no table for entity_CostCenter_source created. It is because either the new table didn’t get created or you used an old pipeline with a new entry for entity_CostCenter_source without overriding existing tables. Our internal engineering team suggests deleting all existing tables and views, creating a new pipeline, and re-run the job.
Comments
Please sign in to leave a comment.