1. Reltio Support
  2. Knowledge Base
  3. Pipeline

How can I enable Snowflake connector activity logging in Snowflake connector external Staging?

Gloria
Updated

Summary

This article explains how activity logging works for the Reltio Snowflake Connector in the two supported integration patterns:

  • Legacy External Storage (a.k.a. “External Storage mode”): Reltio exports data to cloud storage; the connector ingests from there into Snowflake. Logging is handled outside Snowflake by Reltio’s integration layer and/or your cloud analytics sink.
  • Direct Connect: Reltio communicates with Snowflake via Snowflake APIs. Logging is primarily handled inside Snowflake using native telemetry (query history, access history, monitoring views). Reltio may also expose connector-level logs, but the configuration differs from the legacy mode.

You can use this guide to decide where to find logs, how to enable them, and how to access/monitor them in each mode.

Prerequisites

  • You know which connector mode your tenant uses (Legacy External Storage vs Direct Connect).
  • Access to your cloud storage/analytics sink (e.g., S3, Azure Blob, GCS, or BigQuery) for legacy mode, and appropriate Snowflake roles to view account usage in direct mode.

Modes at a Glance

Area Legacy External Storage Direct Connect
Data path Reltio → External storage (S3/Azure Blob/GCS) → Snowflake Reltio ⇄ Snowflake (APIs)
Primary log source Reltio integration layer & your cloud sink Snowflake native telemetry (query/access history)
Control toggle activityLogEnabled in Reltio config Snowflake-native; Reltio log config may differ or be unnecessary
Log location Cloud storage and/or analytics platform (e.g., BigQuery) Snowflake ACCOUNT_USAGE / INFORMATION_SCHEMA views
How to access Cloud console/bucket or analytics UI Snowflake UI, Snowsight, or SQL
Typical use cases Monitor exports/imports & connector moves across cloud storage Audit connector queries, performance, and access within Snowflake

Legacy External Storage Mode

In Legacy External Storage mode, the connector’s activity logs are produced by Reltio’s integration layer and land in your cloud sink (S3 / Blob / GCS or an analytics warehouse like BigQuery). Snowflake is only the destination for the exported data, not the activity log stream—so there’s no required “activity” table in Snowflake for setup.

Common gotchas

  • Expecting to see legacy activity in ACCOUNT_USAGE—you won’t; that’s Direct Connect territory.

  • The activityLogEnabled toggle applies to legacy logging in your cloud sink, not to anything inside Snowflake.

How it works

  1. Reltio does not connect directly to Snowflake.
  2. Reltio exports data to an external storage location (e.g., S3, Azure Blob, GCS).
  3. The Snowflake connector loads from external storage into Snowflake.

Activity logging behavior

  • When activityLogEnabled=true in the Reltio configuration, the integration layer generates activity logs (e.g., export/import events).
  • Logs are external to Snowflake. They are either:
    • Written alongside exported data in your cloud storage, and/or
    • Streamed to an analytics platform per your configuration.

In the legacy model, activity logging is commonly discussed as an external capability. However, if you want a Snowflake-resident Activities dataset for processing connector activity events from the landing table, the deprecated legacy scripts provide that path.

Legacy external staging: scripts used for the Activities table

For legacy external staging, the following scripts are the relevant building blocks for creating and populating the Activities table.

Script namePurposeRequired placeholderscommon.landing-tableCreates the landing table that receives staged JSON ingestiondatabase, schema, landingTableNameactivities.streamCreates the Activities stream on the landing tabledatabase, schema, landingTableNameactivities.staging-tableCreates the Activities staging tabledatabase, schemaactivities.processing-taskCreates the task that merges activity data from the stream into the Activities tablewarehouse, database, schema
 

Why all four matter: The landing table is the ingestion source, the stream captures changes from that landing table, the staging table stores the structured activity records, and the processing task moves and merges the data into the target structure.

How to use the legacy scripts

You can retrieve the SQL for each script via the Adapter Script API, then execute the rendered SQL in Snowflake by replacing the required placeholders with values from your environment.

Recommended execution order:

  1. Create the landing table with common.landing-table.

  2. Create the Activities target table with activities.staging-table.

  3. Create the stream with activities.stream.

  4. Create the processing task with activities.processing-task.

  5. Resume or enable the Snowflake task if your operating model requires manual activation.

  6. Validate that new landing-table records are being captured and merged into the Activities table.

Best practice: Use a dedicated warehouse for connector processing tasks so activity ingestion does not compete with analyst workloads.

 

Retrieving script content from the Adapter Script API

The script catalog states that script content can be retrieved through the Adapter Script API. Use the script name as the name parameter.

Example API calls:

POST {{hub-url}}/api/tenants/{{tenantId}}/adapters/{{adapterName}}/scripts?name=common.landing-table
POST {{hub-url}}/api/tenants/{{tenantId}}/adapters/{{adapterName}}/scripts?name=activities.staging-table
POST {{hub-url}}/api/tenants/{{tenantId}}/adapters/{{adapterName}}/scripts?name=activities.stream
POST {{hub-url}}/api/tenants/{{tenantId}}/adapters/{{adapterName}}/scripts?name=activities.processing-task

After retrieving the SQL:

  • Replace each placeholder with your actual Snowflake values.

  • Run the scripts in Snowflake in the recommended order.

  • Confirm that the task can access the landing table, stream, and target table in the selected database and schema.

Operational flow in legacy external staging

Once configured, the activity pipeline works like this:

  1. Reltio writes exported connector payloads to external storage.

  2. The Snowflake connector loads staged data into the landing table.

  3. The activities.stream object tracks new change records on that landing table.

  4. The activities.processing-task task reads from the stream and merges activity data into the Activities table created by activities.staging-table.

The legacy scripts make it possible to maintain an Activities table in Snowflake even though the legacy logging model is generally described as external to Snowflake.

Current equivalent objects in the newer approach

If you are comparing the legacy setup to the current architecture, these are the newer script names for the Activities dataset:

Current scriptPurposedb.tables.create-table-activitiesCreates the activities tabledb.streams.create-stream-activitiesstreamCreates the activities streamdb.tasks.create-task-activitiestaskCreates the activities processing taskdb.views.create-view-activeactivitiesCreates the active activities view

Recommendation: For new implementations, prefer the current script family and setup automation. For existing legacy external staging environments, the deprecated scripts remain the relevant way to create and populate the Activities table.

Where to find logs

  • Cloud storage: Look under the designated bucket/container and prefixes your team configured for connector runs.
  • Analytics sink (e.g., BigQuery): Use your standard dataset/tables created for connector activity logs.

Operational notes

  • Retention & cost: Govern via your cloud storage lifecycle rules or analytics warehouse policies.
  • Alerting: Use your cloud-native tools (e.g., CloudWatch, Azure Monitor, GCP Monitoring) to alert on missing files, error flags, or anomalous counts.

Direct Connect Mode

How it works

  1. Reltio connects directly to Snowflake using Snowflake APIs.
  2. Data is read/written without external storage in the middle.

Activity logging behavior

  • Logging is primarily internal to Snowflake. Use Snowflake’s native telemetry to audit and monitor connector activity:
    • QUERY history for executed statements
    • ACCESS history for data object access
    • Additional account usage and monitoring views
  • Reltio may still expose connector-level logs; however, the configuration and availability differ from legacy mode and might not rely on activityLogEnabled.

Where to find logs in Snowflake

  • Snowsight / Web UI: Activity → Query History / Access History
  • SQL (examples; adjust filters to your environment):

Reference documentation

Minimal Configuration Reference

Legacy External Storage

  • Toggle: activityLogEnabled=true
  • Destination: Cloud storage and/or analytics platform (e.g., BigQuery) configured by your team
 
{
"activityLogEnabled": true
// Additional environment-specific settings define where logs are written
}

Direct Connect

  • Primary telemetry: Snowflake history and monitoring views
  • Reltio logs: May be available; refer to your connector configuration for options

Summary Table

Feature/Mode

External Storage (Legacy)

Direct Connect

Data Transfer Path

Reltio → External Storage → Snowflake

Reltio ↔ Snowflake (direct)

Activity Log Location

External (e.g., BigQuery, S3, etc.)

Snowflake native logs

Enable Logging

activityLogEnabled in Reltio config

Snowflake logging/monitoring tools

Log Access

Cloud storage or analytics platform

Snowflake UI/SQL

 

 

 

 

 

 

 

 

 

 

Was this article helpful?
0 out of 0 found this helpful

Comments

0 comments

Please sign in to leave a comment.

Articles in this section

See more