Why is my export failing with a "Invalid stream length, the length must be less than or equal to 100 MB in size." error in Azure?

Question

 

We are unable to export data from Reltio using the Export feature using Azure.  We see the following error - Invalid stream length, the length must be less than or equal to 100 MB in size.

 {
"id": "9e30807c-8d36-4cb2-b713-d80bca0f68db",
"groupId": "9f60dc7c-6581-44f5-b11f-6a5db5f6f15f",
"createdTime": 1641289773070,
"type": "com.reltio.export.tasks.local.es.EntitiesESExportTask",
"status": "FAILED",
"name": "Reltio_Export_Consumer_1",
"createdOnHost": "export-7f56dd6bcd-bhdws",
"executedOnHost": "export-7f56dd6bcd-jnz7j",
"parallelExecution": false,
"nodesGroup": "etalon-prod-h360-jobs",
"startTime": 1641289804924,
"endTime": 1641292245740,
"parameters": {
"tenantId": "5fz3GlLwNMyYQBp",
"distributedTaskIndex": "0",
"distributedTasksPartsCount": "2",
"taskId": "5fz3gllwnmyyqbp_09-49_entities_57f0",
"exportName": "5fz3gllwnmyyqbp_09-49_entities",
"exportPath": "",
"fileFormat": "CSV",
"dateFormat": "READABLE",
"ovOnly": "true",
"exploded": "true",
"encryptUrl": "true",
"useEsHeadersCollectionExport": "false",
"exportType": "ENTITIES",
"activeness": "ALL",
"headersStyle": "name",
"sePrefix": "false",
"skipPostprocessing": "false",
"fileCompression": "ZIP",
"headerFormat": "DEFAULT",
"azureStorageContainer": "reltio",
"azureStoragePath": "reltio_csv",
"currentState": {
"lastExecutionEC2Instances": "i-0732f6a50d6555ed1",
"numberOfFailedToPublishEvents": 0,
"globalStatus": {
"headerTotal": 1388443,
"lastFiveMinReadThroughput": 375,
"lastFiveMinThroughput": 0,
"eta": "38m 30s",
"entitiesTotal": 1388443,
"headerNumber": 1388443,
"entitiesNumber": 0,
"overallThroughput": 601,
"overallReadThroughput": 803,
"etaMillis": 2310000
            },
"lastHourThroughput": 350.92889404296875,
"statusId": 60,
"numberOfProcessedObjects": 0,
"numberOfObjects": 87450,
"storageType": "AZURE",
"status": "Exporting..."
        },
"throughput": 0.0,
"duration": "40m 40s",
"error": "Invalid stream length, length must be less than or equal to 100 MB in size."
    },

 

Answer

The recommendation is to execute the export API and please apply the "partsize" parameter (reference https://docs.reltio.com/export/exportentities.html)

 

Was this article helpful?
0 out of 0 found this helpful

Comments

0 comments

Please sign in to leave a comment.