Question1Replies31Views Sivajyothi Member since 2015 11 posts HSBC Software Solution Posted: 6 months agoLast activity: 6 months ago Kafka Stream DataFlow consuming Old Data at a Particular time everydayHi ,We are observing an unusual behavior of Kafka stream data set every day at particular time(between 23:00 - 00:00) where it is reading all the records of kafka topic again.We are connecting to exterenalkafka system and the Kafka configirations are done as below at Pega:- configured Kafka Instance to connect to external Kafka server- configured Kafka stream data set - configured Real time data flow with Kafka stream data set as source and with 'only read new records' optionExample scenario: - Kafka Topic got 100 records in a particular day and pega reads them as and when they are available in Kafka topic.- At end of the day(between 23:00 - 00:00) , again reading all those 100 records. Pega Version : 8.1.7Server : WAS Liberty.Please provide any inputs .Thanks in advance. Pega Marketing 8.1 Low-Code App Development Dev/Designer Studio Data Integration Decision Management Conversational Channels Other Industry ×Close popoverFacebookTwitterLinkedinEmail Copy Link Copied! Posted: 6 months agoKevinZheng_GCS PEGA replied to SivajyothiThis is very interesting. How did you know that the same 100 records were consumed again at 23:00? By turning on org.apache.kafka log? Can you attach the logs if possible?