Queue processor stream are duplicating reprocessing of kafka stream at retention
We are using Queue processor methodology - dedicated queue for real time event push to reporting system. We didn't had duplciate issue with lagecy "connect queue" agent processing, but queue processor kafka we are seeing issue "same message is coming to processing again after couple of hours [even it had processed prior] ".
Please help to resolve this duplicate execution every day we are seeing ~ 250 messages; looks like it happen at retention cleanup time.( I think so)
it hard to trace queue processor in 1.2 -1.5 million event and pattern i am seeing after 8 -12 hours at might duplication
we have observed the similar issue where we have queued items being processed multiple times. Along with this issue , we noticed that the Stream node is going down and due to which the Queue Processor execution is failing - giving PEGA0072 - DataFlow.backgroundProcessing service is failed.