Question

1
Replies
4940
Views
sabaf985 Member since 2013 9 posts
Cognizant Technology Solutions
Posted: November 11, 2015
Last activity: March 24, 2016
Closed
Solved

How to handle large data (around 100,000 records) in a file using a file listener.Is there any best option to achieve the same?

My requirement is as follows,

I have a file with large data around 100,00 records in a file. I need to take each record and verify in a DB table.If the record exists in that table,I need to update the record in the DB with the values which I get in the file ,if the record does not exist I need to insert a new record in the DB table.

Do I need to handle this in the listener activity itself or is there any best option to achieve this?

Also If there occurs any error while processing the file, say the listener has processed 1000  records successfully and it failed for 1001 record then how to handle it as we will be getting the file only once..

Thanks & Regards,

Saba Fathima

Data Integration
Moderation Team has archived post
Share this page LinkedIn