Posted: 13 Dec 2019 15:44 EST Last activity: 11 Jun 2020 16:23 EDT
Tracing Techniques for the Pega Data Flows
The attached document is the lesson learned for a future reuse with the Data Flow tracing techniques. It might be useful to see how to run the Pega Tracer to trace the execution of the Pega Data Flow run, or how to see the values populated on the records, while going thru the Data Flow run.
“Data flows allow you to sequence and combine data based on various sources and write the results to
a destination. The sequence itself is established through a set of instructions and execution points from source to destination. Source and destination points can be abstract or driven by data sets and other decision data flows. Between source and destination, you can apply compose, convert, and merge strategy execution instructions.
The execution of data flows is done through work items which are instances of Pega-DM-DDF-Work and stored in the pc_work_dsm_batch table. The processing of data flows can be divided in two categories:
Batch data flows using a database table as main input.
Real-time data flows that, in active state, continue processing incoming stream data and requests made through the available service interface.
Data flow runs that are initiated through the Data Flows landing page run in the access group context. These data flows always use the checked-in instance of the Data Flow rule and the referenced rules. You can use a checked-out instance of the Data Flow if you initiate a local data flow run (by using the Run action in the Data Flow rule form) or a test run (a run initiated through the API).”
I am exactly looking to trace the Data flow execution and for that calling the data flow with browse operation. I am getting the results in the results page but I dont see any of the Data flow execution steps in the tracer.
Any help would be greatly appreciated.
Basically I need to refer to the Run ID during the data flow execution and am not finding anything to get to that run ID during the data flow execution.
Posted: 2 years ago
Posted: 26 Mar 2020 8:49 EDT
Pawel Nowak (nowap)
Director of Service Assurance
You can create an Activity to run the Data Flow in the browse mode (not in start mode) and run it with tracer. The browse mode will execute the Data Flow using your requestor session, so you will see the execution of that Activity in the tracer. You can also run a tracer on a particular rule from the Data Flow, like Data Transform, so the tracer will not be very long.
The run ID can be either set manually to be a constant value like RUN-1, so every time, when the Data Flow run is executed, it reuses the same run ID, but having that, Pega overrides a history of that run, or the run ID can be set dynamically, so every execution of the Data Flow run gets a new run ID and Pega keeps all the Data Flow run history for analysis. When you execute the Data Flow run, either form the Data Flow run Landing Page, or from an Activity, or from a Data Flow using the Action-Run option, Pega creates an internal case for each execution of that Data Flow run and there are internal OOTB Pega classes, where you can find the run ID too, like Data-Decision-DDF-Progress, Data-Decision-DDF-RunOptions
This might be not so easy, but you can invoke the same flow with static run ID multiple times. Why you need to have dynamic run ID? Basically data flow run works with pyWorkPage, as it is an internal case to execute the flow. How you trigger that data flow run, is it triggered from activity?