How can I create a DataBricks token to be used in the Reltio DataBricks DPH connector?

Create the DataBricks token in DataBricks

1. Log in to your Databricks workspace using the appropriate link for your cloud provider:

   • Azure: Databricks - Sign in

   • AWS: Databricks - Sign in

   • GCP: Databricks - Sign in

2. Navigate to Access Tokens:

   • Go to Settings > Developer > Access Tokens > Manage (please reference DataBricks documentation in https://docs.databricks.com/aws/en/dev-tools/auth/pat#databricks-personal-access-tokens-for-workspace-users)

3. Generate a New Token:

   • Click Generate New Token.

   • Name the token (e.g., dph-databricks-<<ENV>>).

   • Set the token lifetime (e.g., 730 days, equivalent to 2 years).

   • Click Generate.

4. Save the Token:

   • Copy and securely save the generated token immediately. Unfortunately, you won't be able to view it again.

5. Copy the Host Name:

   • The host name is the URL of your Databricks workspace (e.g., adb-xxxxxx.azuredatabricks.net).

6. Use the Token:

   • You can use the generated token and hostname in your connector or application configuration as required.

Store the Token Securely

• If your setup uses a secret manager (recommended), you can just store the token there and reference it in your configuration.

• For example, you may use an API call using POSTMAN to store the token:

PUT https://<dph-url>/api/tenants/<tenant-name>/adapters/<adapter-name>/secrets 

Request body
{
  "Databricks": {
    "token": "<DATABRICKS_TOKEN>"
  },
  "Azure": {
    "sasToken": "<SAS_TOKEN>"
  }
}

This ensures the token is not exposed in plain text in configuration files.
 

Note:

• The token must have sufficient permissions for the operations you intend to perform (e.g., managing DLT pipelines, workspace folders, and Unity Catalog).

• For service principals, tokens must be created using the Databricks Token Management.

Example of DPH configuration in physical tenant:

{
  "type": "deltalake",
  "enabled": true,
  "cloudProvider": "AZURE",
  "azureConfig": {
    "azureCredentials": {
      "authMethod": "TOKEN",
      "storageAccountName": "<STORAGE_ACCOUNT>"
    }
  },
  "stagingBucket": "<STAGING_CONTAINER>",
  "databricksConfig": {
    "databricksHost": "https://<your-workspace>.cloud.databricks.com/",
    "databricksCredentials": {
      "authMethod": "TOKEN"
    },
    "secretsScope": "AzureSecrets",
    "azureConfig": {
      "queueName": "<QUEUE_NAME>",
      "subscriptionId": "<SUBSCRIPTION_ID>",
      "stagingStorageAccountCredSecretName": "<SECRET_NAME>",
      "stagingStorageAccountConnectionStringSecretName": "<SECRET_NAME>"
    },
    "catalog": "<CATALOG_NAME>"
  }
}