Posted: 5 May 2017 6:47 EDT Last activity: 12 Jul 2017 7:16 EDT
How to find the completion of Data Flow called from Activity
We have a Dataflow called in an activity using DataFlow-Execute method, The activity calls the Dataflow in one step and before the Dataflow gets completed, the activity moves to the next step whereas the requirement is the activity should wait till the Dataflow completes the job and then activity should move to the next step.
How can we determine the dataflow completion status or is there any better way to deal this requirement?
***Updated by moderator: Lochan to add Categories***
As I suggested in a comment below, using Thread.sleep() is a risky workaround. What if you get to step #5 and the data flow hasn't finished yet? Consider using postActivity that queues another activity and continues processing. You can then load the progress of the data flow via DataFlow-Execute activity method and use "Get Progress" operation giving the run ID.
Posted: 4 years ago
Posted: 12 Jul 2017 7:16 EDT
Oguz Cavli (cavlo)
Principal Decisioning Architect
There's currently no out of the box way of waiting for a batch data flow run completion, you would need to do that yourself. Usually batch data flow runs last for a very long time and because of that it is executed in a different thread so the current one is not blocked.
An alternative I can suggest is to use PostActivity processing. Once the data flow data processing is finished the configured activity will be triggered. Keep in mind that in this case the post activity will be executed in a different thread and the status of the run at that point will be "Running post activity".
If you need to wait for the data flow run to finish in the same thread you will need to implement the wait mechanism yourself. You can do that by invoking DataFlow-Execute GetProgress operation and verifying the value of pyStatus in the progress page. It should be either "Completed" or "Failed" once finished.