Please use Data Flows that is available as part of Decisioning framework. Nowerdays decisioning is a part of Pega only so you should already be having it. By the way what are you trying to achieve from that 2.5 million records?
We are retrieving 2.5 million records, checking some business conditions, creating cases and saving in different tables. around 1.5 million records may get dropped in process and will be creating cases for 1 million records.
But, here we are not Decision framework, so cannot use data flows.
if your report will retrieve a large number of records, definitely turn on "paging" on the report. Sometimes, even though your db is 2.5 million records, you don't intend to retrieve many at a time. Reduce the number of columns you retrieve until you get a sense of the performance. /Eric
I hope I understand your requirement. You to need read records from DB table, perform some business logic & repeat this process till some record count reaches or some condition is met.
If this is correct then why don't you to try Agent which execute one activity with Obj-open/browse , perform business logic on row(s) and configure agent to repeat after some time. Few merits to this solution.
You don't need to initiate the process by yourself. It is automatic as per configuration
Process will run in the back end so no human effort to run this.
Since you need to process few million records so if you do it at one short then it might cause performance issue in system. But agent will do it in sequence with only few records at one short.
You can choose the time to perform this job, when system is less used for business.