Why is the data load completed job display showing more profiles loaded than what was actually in the file to be loaded?

Question

You are uploading the files into S3 (because the file size is more than 50MB) and you are loading the data from Console. After completing the first file the console is displaying the records of all the respective files, please check the below details.

Example1:

Files:
FSC_Reltio_Consumer_PhoneAttributes – 235,517 Records (but in the load, it is displayed as 1,135,517 records which sum of all the below files).

FSC_Reltio_Consumer_PhoneAttributes1 – 300,000 Records
FSC_Reltio_Consumer_PhoneAttributes2 – 300,000 Records
FSC_Reltio_Consumer_PhoneAttributes3 – 300,000 Records

Example 2:

Files: 
FSC_Reltio_Consumer_EmailAttributes – 229,059 Records but in the load, it is displaying it as 529,059 records. 
FSC_Reltio_Consumer_EmailAttributes1 – 300,000 Records
First Load is displaying the records for all the files

Example 3:

Files: 
FSC_Reltio_Consumer_Attributes – 214,182 Records but in the load, it is displaying it as 1,414,182 records. Please check the below screenshot.

FSC_Reltio_Consumer_Attributes1 – 300,000 Records

FSC_Reltio_Consumer_Attributes2 – 300,000 Records

FSC_Reltio_Consumer_Attributes3 – 300,000 Records

FSC_Reltio_Consumer_Attributes4 – 300,000 Records

Again, the first Load is displaying the records for all the files

Answer

Here are the different files that you are loading for one set of jobs mentioned in Example 3

Job Definition 1 :/FSCtoReltio/12232021/FSC_Reltio_Consumer_Attributes
Job Definition 2 :/FSCtoReltio/12232021/FSC_Reltio_Consumer_Attributes1
Job Definition 3 :/FSCtoReltio/12232021/FSC_Reltio_Consumer_Attributes2
Job Definition 4 :/FSCtoReltio/12232021/FSC_Reltio_Consumer_Attributes3
Job Definition 5 :/FSCtoReltio/12232021/FSC_Reltio_Consumer_Attributes4

As you can see they all have common prefix /FSCtoReltio/12232021/FSC_Reltio_Consumer_Attributes which is the same as the file Job Definition 1 is supposed to load, and the provided source path is not the complete name of the file (which actually is FSC_Reltio_Consumer_Attributes.csv)

Since you have provided a prefix, not the complete name of the file to load, Job Definition 1 listed all the files with the provided prefix and found 5 files to load hence the count is higher than expected(which is equal to the sum of all records in 5 files combined).

If you wish to load files with common prefixes separately, the console data loader needs the full name of a single file to load or avoid the common prefixing.

Keep in mind that there are file size limitation

• The file size limit allocated for a .csv file is 33GB. Make sure the file size does not exceed 50MB when you upload a file from the local file system.
• The file size limit allocated for a JSON file is 50MB, irrespective of whether you upload the file from the local file system or remote storage.