We are using Queue processor methodology - dedicated queue for real time event push to reporting system. We didn't had duplciate issue with lagecy "connect queue" agent processing, but queue processor kafka we are seeing issue "same message is coming to processing again after couple of hours [even it had processed prior] ".
Please help to resolve this duplicate execution every day we are seeing ~ 250 messages; looks like it happen at retention cleanup time.( I think so)
it hard to trace queue processor in 1.2 -1.5 million event and pattern i am seeing after 8 -12 hours at might duplication
we have observed the similar issue where we have queued items being processed multiple times. Along with this issue , we noticed that the Stream node is going down and due to which the Queue Processor execution is failing - giving PEGA0072 - DataFlow.backgroundProcessing service is failed.
@BHUSHANPATIL We are also facing this issue where a dedicated queue processor with immediate processing which runs at 12 midnight ET is running multiple times. We have started seeing this issue after daylight time changes from EST to EDT. We process 100k items with these queue processor per day and duplicate processing of same queue processor item is causing a lot of issues.