Posted: 30 Dec 2020 11:41 EST Last activity: 30 Dec 2020 11:45 EST
Reprocessing failed records/partitions in batch data flow runs
Hi, we have a use case where we need to rerun a datflow for failed records from Batch processing. I found the below article to rerun the data flow for failed partitions( which can have failed and successful records).
While rerunning, it processing both failed and successful records from partition. Because of that in Batchout table we could see some duplicate records. Is there anyway to avoid processing success records from failed partition? or remove the Successful partition records from Batchout table before reprocessing? Please advise.