General description

Survivorship rules are used to determine Operational Values (OV) for object (an entity/relation)  attributes. Each attribute can have 0, 1, or multiple values that have been marked as OV, they are also called “winner values”.

The changed module calculates those values.

Survivorship rules are defined for object attributes by survivorship mappings. All rules are defined separately for each attribute and are calculated independently of each other. A survivorship group consolidates a set of survivorship mapping.

An entity type can have multiple survivorship groups: one of them must be marked as default. A default group is used in most cases.

Other, not default groups are applied for users with specific roles only(roles are also defined in a group setting).

If some attribute isn’t specified in NOT default mapping, the mapping for this attribute is taken from the default group.

All survivorship rules are set in a business configuration for object types.

Survivorship calculation rules include:

• A survivorship strategy which applies specific rules to determine winner values;
• Filterings settings that split all attribute values by defined criteria. Each split set of values have its own winner values, all calculation is performed independently between split sets;

• Fallback strategy settings tell what strategy should be used to determine the winner if the main survivorship strategy couldn’t determine them or found a lot of them.

• Restrictions of sources for OV which are set by sourcesForOv field. If these restrictions exist, only crosswalks (and values linked to them) from permitted source-systems are used to determine winners, all other crosswalks and values are ignored. So if there are no values from permitted sources, there are no winner values.

• Pinned value: survivorship rules are not applied to attribute values if one of those values is pinned. All pinned values become OV’s and all attribute’s survivorship rules are just ignored.

• Ignored value: Ignored values are not participating in OV calculation just as those attributes don’t exist.
• Crosswalks with end date (delete date) are not participating in OV calculation just as those crosswalks don’t exist.

• DEFAULT strategy: If an attribute doesn’t have its own survivorship mapping, the default survivorship strategy LUD (Last Update Date)  is applied.

If all crosswalks of value were filtered out because they have an end date or they aren’t’ from sourcesForOv such value is ignored and not participate in ov calculation.

Survivorship strategies

There are 10 survivorship strategies:

1. Last Update Date (LUD). Finds crosswalk with the most recent update date. All values that were provided by this crosswalk become winners.
• If the value has the most recent singleAttributeUpdateDate its crosswalk becomes a winner crosswalk. If there are many values provided by this winner crosswalks, all of them become winners
• If there are many crosswalks has the same update date, only the first crosswalk become a winner
2. Source-System (SRC_SYS). The user provides a priority list of sources. A list of priority is defined for mapping, if it’s absent, it is taken from the priority list of the group. All crosswalks from a source system with the highest priority become winners, all values of those crosswalks also become winners.
• If an attribute doesn’t have values from sources of the priority list, the priority of sources is taken from the source types configuration of the business model.
• If strategy applied to one value only, this value and its crosswalk become winners, irrespective of whether this crosswalk has sourced from the priority list or not.
• If there are many values with crosswalks has no priorities for sources, the winner value becomes those value which has the most recent Cassandra update time. All crosswalks of this value become winner crosswalks.
3. Frequency. Calculate amounts of crosswalks from which values were provided. Values with the most amount of crosswalks become winners.
• If more than one value is the winner, the LUD strategy is used to find final winners.
4. Aggregation. All presented values become winners.
5. Record-Values Based Source Priority. All values from the winner source type become winners.
• WHOLE object’s attributes checked if some value from values of winnerSourceAttributes has crosswalk with a winner source type. In this case, all values and theirs crosswalks with this source type become winners.
• If the WHOLE object doesn’t have values from the winner source type, LUD strategy is applied instead.
6. Custom Survivorship Strategy. All crosswalks from the winner source type become winners, all values from those crosswalks also become winners. Contains a parent strategy in the definition.
• If several or no crosswalks win, the parent strategy is applied to determine the winner.
7. Oldest Value. Finds crosswalk with the oldest create date. All values that were provided by this crosswalk become winners.
• If there are many crosswalks has the same creation date, only first crosswalk become a winner

  8. Min Value. Values with the smallest value becomes winners. All crosswalks of those values become winners.

• • If this strategy applied to nested attributes (nested attribute doesn’t have value for comparison) it should be specified comparisonAttributeUri . If comparisonAttributeUri is not specified or the value of this attribute doesn’t exist, the value of the nested attribute considered as NULL. It’s always counted as the most maximum than any other value.

 9.  Max Value. Values with the highest value become winners. All crosswalks of those values become winners.

• • If this strategy applied to nested attributes (nested attribute doesn’t have value for comparison) it should be specified comparisonAttributeUri . If comparisonAttributeUri is not specified or the value of this attribute doesn’t exist, the value of the nested attribute considered as NULL. It’s always counted as most minimum than any other value.

10.  Other Attribute Winner Crosswalk. Gets all winner crosswalk from the primary attribute. If the current value has those crosswalks, all of them and all values provided by them become winners.

Some strategies use simple attribute values to define winners (Min Value, Max Value, Frequency). But simple values don’t exist for nested and referenced attributes, so to apply those strategies to nested and referenced attributes an attribute for comparison should be specified. It is taken from comparisonAttributeUri property of a survivorship mapping. A comparison attribute must be sub-nested (sub-referenced) for calculate attribute, ALL values of comparison attribute are taken for comparison.

Filtering

Filtering settings split all values from an attribute by filter on different sets of values. Those sets of values can intersect each other if a value meets conditions for different filters. For each of those sets, winners are calculated separately. Each filtering condition specified in separate attribute mapping. If some value contained in many subsets it’s enough to become the winner in one of them.

If a value doesn’t match any filter from mappings, it goes to the set which will be calculated by default strategy and also separately from other sets. This strategy should be specified separate mapping without filtering property otherwise LUD will be applied.

Filtering supports only “equal” expression. As a URI for matching any attribute can be used, for example:

• A value from a neighbor attribute on the same level (FirstName -> LastName, Identifiers/ID -> Identifiers/Type).
• Not neighbor value from anywhere (Identifiers/ID -> Address/AddressLine1).

If multiple values exist on URI, only one required value is enough for the filtering match.

Fallback strategies

Fallback strategy options are determined by “fallbackUsingCriteria” and “fallback strategies” properties on a survivorship mapping.

There are 2 fallbacks using criteria:

• MORE_THAN_ONE (default). If the main strategy returns 2 or more winner values, a fallback strategy chain will be applied to winner values step by step until 1 or 0 values remain. ONLY winner values and ONLY winner crosswalk from PREVIOUS steps are taken into account.
• ZERO_OR_MORE_THAN_ONE. If the main strategy returns 0, 2 or more winner values, a fallback strategy chain will be applied to winner values step by step until 1 value remain. ONLY winner values and ONLY winner crosswalk from PREVIOUS steps are taken into account.

Example. Let’s suppose we have the following mapping:

• Main strategy: Other Attribute Winner Crosswalk.
• Fallback strategies:
• Record-Values Based Source Priority with winner source type A.
• Record-Values Based Source Priority with winner source type B.
• Max Value.

Then we start to calculate winners step by step.

1. We apply Other Attribute Winner Crosswalk. It returns 5 winners.
• If the fallback using criteria is MORE_THAN_ONE, we take 5 winners and filter their crosswalks to use only winner crosswalks. Then use them on the next step
• The same thing for ZERO_OR_MORE_THAN_ONE criteria.
2. We apply Record-Values-Based Source Priority (source A). It returns 0 winners.
• If the fallback using criteria is MORE_THAN_ONE, we stop the calculation. The result will be 0 winners.
• If the fallback using criteria is ZERO_OR_MORE_THAN_ONE, we take the previous 5 winners with their winner crosswalks and use them on the next step.
3. (For ZERO_OR_MORE_THAN_ONE only) We apply the second Record-Values-Based Source Priority. Let’s watch different results:
• If it returns 0 winners, we take and use the 5 previous winners again on the next step.
• If it returns 1 winner, we mark it as the final winner and stop the calculation.
• If it returns 2 or more winners, we take them and use them on the next step.
4. (For ZERO_OR_MORE_THAN_ONE only) We apply Max Value. As this is the last strategy in the chain, no matter what the result will be, it will be used as the final result (with 0, 1 or more winners) anyway.

NOTE: Fallback strategy chains use not only winner values but also winner crosswalks of those values. So if a value with multiple crosswalks becomes a winner on a calculation step with just a few winner crosswalks, only those winner crosswalks are used on the next calculation step.

Interfaces

The next interfaces are related to OV strategies:

IOVStrategy. The main interface of all OV strategies.

ICrosswalkBasedOVStrategy. The interface of strategies that use crosswalk properties (update date, source type, etc.) to determine winners. Strategies: Aggregation, LUD, Oldest, Other Attribute Winner Crosswalk, Record-Values Based, SRC_SYS.

• Collection<AttrCrosswalkWrapper> calculateWinners(Collection<AttrCrosswalkWrapper> crosswalks, ICalculationInfoProvider infoProvider) - returns winners;
• boolean useCrossConnectedCrosswalksForReferenced() - returns true if a strategy should use entities crosswalks for referenced attribute values to determine winner (instead of default using of relation crosswalks).

IValueBasedOVStrategy. The interface of strategies that use values to determine winners. Strategies: Min Value, Max Value.

• Collection<AttrValueWrapper> calculateWinners(Collection<AttrValueWrapper> values) - returns winners.

ICrosswalkValueBasedOVStrategy. The interface of strategies that use both crosswalks and simple values to determine winners. Strategies: Custom, Frequency.

• Collection<AttrCrosswalkValueWrapper> calculateWinners(Collection<AttrCrosswalkValueWrapper> crosswalkValues, ICalculationInfoProvider infoProvider) - returns winner.

Other useful interfaces:

• IOVStrategyExecutor - an interface to which execution of different types of OV strategies is delegated. Makes repacking of a set of calculated values to different values and calls a required strategy.

• IOVStrategyFactory - an interface that creates survivorship rules from mappings.

• IExternalAttributesInfoProvider - an interface that provides information about external attribute properties that can be used during calculation (winner crosswalks of another attribute for example).

Calculation algorithm and sequence

• AttributesTO instance that contains information about a set of object’s attributes arrives to OVCalculator class. The class:
• • Extract all values of attributes from all levels.
  • Calls IOVStrategyFactory to create survivorship rules for all attributes that are presented.
  • Determines an order of attribute calculation (for strategies that depends on results of other attributes, Other Attribute Winner Crosswalks for example).
  • Delegates calculation of OV for specific attributes to AttributeOVCalculator class.

• AttributeOVCalculator class calculates winner values for the passed set of attribute values and the list of survivorship rules.

• Check that the passed values are still available for calculation (their crosswalks are not outdated for example).
• Splits the passed values by filters from the list of survivorship rules.
• Extract values from nested/referenced attributes if a strategy requires this (comparisonAttributeUri for Max Value, for example)
• For each filtered set of values performs a calculation of winners via IOVStrategyExecutor.
• Applies fallback strategies if the result of the main strategy does not match acceptance criteria.
• Marks all passed values as OV or non-OV.
• Stores values of winner crosswalk for future possible usage by other attributes and strategies.