Question
I want to know if multiple jobs can be run in parallel: multiple inbound jobs for same or different entities, inbound and outbound jobs running concurrently, outbound jobs for same or different entities, multiple data load jobs including for relationships between entities.
Answer
Yes, Reltio supports running multiple data jobs in parallel—including inbound and outbound jobs for the same or different entity types, as well as data load jobs involving relationships. However, running multiple jobs concurrently can impact system performance, depending on the number of processes and the system load.
To optimize performance during parallel execution, Reltio recommends using the distributed=true parameter. This enables distributed mode, which allows tasks to be executed in parallel across different API nodes. Additionally, you can specify the taskPartsCount parameter to define the number of parallel tasks.(support.reltio.com)
Example: Reindex Task in Distributed Mode
POST {ApplicationURL}/api/{tenantId}/reindex?distributed=true&taskPartsCount=4
In this example, the reindex task is divided into four parts, each executed in parallel. This approach can be applied to various tasks, including data loads and exports.
Additional Considerations
- System Resources: Running multiple jobs in parallel increases the demand on system resources. Monitor system performance to ensure that concurrent jobs do not overwhelm the system.
- Job Prioritization: Reltio processes jobs based on priority and creation time. You can set job priorities to manage the execution order of concurrent jobs.(docs.reltio.com)
- Best Practices: Avoid updating the same objects from different parallel threads to prevent conflicts. Implement retry mechanisms with exponential backoff for failed requests.(support.reltio.com)
For more detailed information, refer to the following resources:
Comments
Please sign in to leave a comment.