Question
Data Flows - Chaining of Data Flows
We had a requirement to execute a data flow based on success or failure of another flow.
So, the first data flow will process a million records and will segment them. Upon completion of segmentation another data flow has to be triggered to route work to appropriate channel.
Problem Statement:
There are two data flows DATAFLOW1 and DATAFLOW2.
DATAFLOW2 must be triggered automatically after completion of DATAFLOW1. This sort of chaining can be extended i.e. DATFLOW3 might have to be triggered based on DATAFLOW2.
I have attached the solution we have implemented. Please let me know if there is any better approach.
We have also explored the option of using intermediate control tables, but we preferred this approach.
Requirement is very simple, trigger another data flow after first data flow is finished. My focus was more on getting the progress page post completion of a data flow.
Once i have the progress page in hand, I can determine success factor or failure factor.
Question was more around, instead of this approach i.e. verifying the progress page in post activity is there any other approach in OOTB?
Hi
one of our strategic application (Pega Marketing) handles such scenario (ie, waiting for DF to complete) by calling a function soon after the dataflow execute call. If you have Pega Marketing installed you could refer ExecuteDDFInBatch activity for ideas.
In the function you could call "Data-Decision-DDF-Progress.pxLoadProgress" activity in a infinite loop and check "Progress.pyStatus" property value, and you break the loop based on the status and exit the function with success or failure.
I have reviewed your design, it appears to be incomplete. Or maybe you've shared only bit and pieces of the core design. I would suggest sharing the design implementation in details.
Are you able to get the desired result?
Design depends on how is your business requirement.
Just a pointer, I couldn't see any iterative loop configured nor conditional "When" to check whether DataFlow1 was successful or not? How you're handling iteration or process?