1. Reltio Support
  2. Knowledge Base
  3. Pipeline

Steps to delete and recreate a Reltio Databricks-datashare using API

Gloria

This article describes how to reset a Reltio Data Sharing with Databricks configuration by deleting an existing Databricks Data Share adapter, recreating the adapter, provisioning the Databricks Data Sharing resources, performing the required initial synchronization, and validating the new Data Pipeline.

This procedure is appropriate when a Databricks Data Share must be rebuilt—for example, when replacing an existing adapter configuration, changing the Data Share name, or reprovisioning the Data Share resources.

Important: Deleting and recreating a Data Share is an administrative operation. Verify the tenant, environment, adapter name, and Databricks sharing identifier before executing the procedure. The source procedure specifically requires confirmation that the adapter being deleted is the intended adapter before issuing the DELETE request.

Prerequisites

Before beginning, collect the following information:

  • Reltio tenant ID
  • Reltio environment hostname
  • Data Pipeline Hub (DPH) hostname
  • Existing Databricks Data Share adapter name
  • New unique adapter name
  • Databricks sharing identifier for the target environment
  • Required ovOnly setting
  • Appropriate authorization for the Reltio and Data Pipeline Hub APIs
  • Environment-appropriate taskPartsCount if distributed synchronization will be used

Do not copy environment-specific values such as Databricks identifiers, timestamps, pipeline IDs, or task identifiers from another environment.

The Data Share name must be unique, and the adapter configuration contains the Databricks identifier and ovOnly configuration.

Procedure

Step 1 — Delete the Existing Databricks Data Share Adapter

First, remove the existing adapter configuration.

DELETE https://<DPH_HOST>/api/tenants/<TENANT_ID>/adapters/<EXISTING_ADAPTER_NAME>

 

Before submitting the request, confirm:

  • <DPH_HOST> belongs to the intended environment.
  • <TENANT_ID> is the correct tenant.
  • <EXISTING_ADAPTER_NAME> is the Data Share adapter that should be removed.

The source procedure uses this operation to remove the previous adapter configuration before creating the replacement adapter.

Step 2 — Create the New Databricks Data Share Adapter

Create the replacement adapter:

POST https://<DPH_HOST>/api/tenants/<TENANT_ID>/adapters

A configuration based on the validated procedure has the following structure:

{
  "type": "datashare-databricks",
  "enabled": true,
  "dataFilteringEnabled": false,
  "sparkJobVersion": "v2",
  "name": "<NEW_UNIQUE_ADAPTER_NAME>",
  "databricksConfig": {
    "identifier": "<DATABRICKS_SHARING_IDENTIFIER>",
    "ovOnly": true
  },
  "createdBy": "<USER>",
  "createdOn": "<CURRENT_UTC_TIMESTAMP>",
  "nativeDeltaProcessing": false
}

Replace all placeholders with values appropriate for the target environment.

In particular:

  • name must contain the intended unique Data Share adapter name.
  • identifier must be the Databricks sharing identifier for the target environment.
  • ovOnly must reflect the required sharing configuration.
  • Do not reuse another environment's createdOn timestamp or Databricks identifier.

The reviewed implementation used a datashare-databricks adapter with sparkJobVersion: "v2" and retained ovOnly: true and the applicable Databricks identifier.

Step 3 — Provision the Databricks Data Sharing Resources

Creating the adapter configuration does not complete the setup. Run the Data Share setup action for the newly created adapter:

POST https://<DPH_HOST>/api/tenants/<TENANT_ID>/adapters/<NEW_ADAPTER_NAME>/actions/setup-databricks-datashare

A successful response should indicate:

{
  "status": "SUCCESS",
  "message": "[<TENANT_ID>/<NEW_ADAPTER_NAME>] Databricks Data Sharing resources created successfully"
}

Do not begin the initial synchronization until this operation completes successfully. The validated procedure explicitly requires a successful setup result before proceeding.

Step 4 — Synchronize the Tenant Data

After provisioning the Data Share, perform the initial synchronization.

The reviewed procedure requires the data types to be synchronized sequentially, completing and validating one synchronization before starting the next.

Use this sequence:

  1. Entities
  2. Relations
  3. Merges
  4. Matches
  5. Interactions
  6. Activities

The general request is:

POST https://<RELTIO_HOST>/reltio/api/<TENANT_ID>/syncToDataPipeline?dataTypes=<DATA_TYPE>&distributed=true&taskPartsCount=<TASK_PARTS_COUNT>&options=parallelExecution

For Entities:

POST https://<RELTIO_HOST>/reltio/api/<TENANT_ID>/syncToDataPipeline?dataTypes=entities&distributed=true&taskPartsCount=<TASK_PARTS_COUNT>&options=parallelExecution

When the request is accepted, Reltio schedules a background task. Capture the createdTime returned by each synchronization request because it can be used to locate that specific task.

Important: taskPartsCount should be determined for the applicable tenant/environment. Values used for one tenant should not be assumed to be Reltio defaults. The source procedure explicitly identifies its task-part values as tenant-specific. Check the physical tenant maxTaskCount to determine the appropriate task count.  Consider other activity that may be occuring on the tenant during a synchronization

Step 5 — Monitor the Synchronization

Use the createdTime returned from the synchronization request to locate the active task:

GET https://<RELTIO_HOST>/reltio/<TENANT_ID>/tasks?showHidden=true&filter=equals(createdTime,<CREATED_TIME>)

Continue monitoring until the task completes and is no longer active. Then retrieve the completed task from task history:

GET https://<RELTIO_HOST>/reltio/<TENANT_ID>/tasks/history?showHidden=true&filter=equals(createdTime,<CREATED_TIME>)

Step 6 — Validate Each Synchronization Before Continuing

Review the task's currentState.

A successful synchronization should have values equivalent to:

{
  "numberOfFailedToPublishEvents": 0,
  "skippedEntitiesCount": 0,
  "errors": [],
  "status": "Completed"
}

Before proceeding to the next data type, confirm:

  • status = Completed
  • numberOfFailedToPublishEvents = 0
  • skippedEntitiesCount = 0
  • errors = []

If skipped objects or errors are reported, stop and investigate before starting the next synchronization.

Repeat the synchronization, monitoring, and validation process sequentially for:

entities
    ↓
relations
    ↓
merges
    ↓
matches
    ↓
interactions
    ↓
activities

The source procedure explicitly requires that each preceding data type be completed and pass validation before the next one begins.

Step 7 — Check Pipeline Status

After recreating the Data Share, retrieve its Data Pipeline status:

GET https://<DPH_HOST>/api/tenants/<TENANT_ID>/pipelines/status

Locate the pipeline associated with:

<TENANT_ID>/<NEW_ADAPTER_NAME>

Confirm that the new pipeline is running and capture its returned:

pipeline_id

Pipeline IDs are environment-specific and generated as part of the pipeline configuration. Do not reuse a pipeline ID from another environment.

Step 8 — Review Pipeline Events

Using the newly returned pipeline_id:

GET https://<DPH_HOST>/api/tenants/<TENANT_ID>/pipelines/<PIPELINE_ID>/events

Review the pipeline state and errors.

A healthy result in the validated procedure included:

{
  "current_state": "RUNNING",
  "latest_update_state": "RUNNING",
  "errors": []
}

In particular, verify:

"errors": []

Step 9 — Review Data Pipeline Event Monitoring

Event processing can also be reviewed through the Data Pipeline Hub Event Monitoring API:

GET https://<DPH_HOST>/api/tenants/<TENANT_ID>/monitoring/_eventMonitoring

Review the results for incomplete or unsuccessful event processing before considering the Data Share reset complete.

 

Troubleshooting

The adapter was created, but the Data Share is not available

Confirm that the following setup action was executed successfully:

POST https://<DPH_HOST>/api/tenants/<TENANT_ID>/adapters/<ADAPTER_NAME>/actions/setup-databricks-datashare

Do not start synchronization until the setup action succeeds.

The synchronization task reports skipped objects or errors

Do not continue to the next data type.

Review:

status
numberOfFailedToPublishEvents
skippedEntitiesCount
errors

The reset procedure requires successful validation before moving to the next synchronization stage.

The expected pipeline cannot be found

Run:

GET https://<DPH_HOST>/api/tenants/<TENANT_ID>/pipelines/status

Confirm that adapter_id corresponds to the newly created adapter and use the pipeline_id returned for that environment.

The Data Share exists, but expected data is missing in Databricks

Verify all three layers:

  1. The applicable syncToDataPipeline task completed successfully.
  2. The new Data Pipeline is running and its events endpoint reports no errors.
  3. _eventMonitoring does not show incomplete or failed processing.

Customer-side Databricks validation should then confirm that the expected shared data is available.

 

Was this article helpful?
0 out of 0 found this helpful

Comments

0 comments

Please sign in to leave a comment.

Articles in this section

See more