What is the difference between when tables are defined as “entities” and “entities_json” in the DataBricks connector?

Question

What is the difference between when tables are defined as “entities” and “entities_json” in the DataBricks connector?

Answer

This is the result of the split_table setting in the physical tenantconfiguration.The entities table and the entities_json table in the Databricks connector serve different purposes and have distinct structures:
Entities Table:

• This table holds the last extracted entity snapshot view.

• It includes columns such as uri, version, timestamp, type, attributes, crosswalks, analyticsAttributes, createdBy, createdTime, updatedBy, updatedTime, committime, deleted, label, linked, secondaryLabel, tags, startDate, and endDate

• The data is structured in a way that provides a comprehensive view of each entity, including metadata and attributes.

Entities_JSON Table:

• • This JSON table typically stores data in a JSON format.

  • JSON tables are often used to store complex, nested data structures that can be easily serialized and deserialized.

  • The structure would differ from the entities table by storing data in a more flexible, schema-less format.

 Setting in the physical tenant

• If splitTable is set to true, data for each entity type is written to separate tables named entity_<entityType>.

• If splitTable is set to false, data for all entities is written to a single table named entities.