Question

1
Replies
31
Views
Sivajyothi Member since 2015 11 posts
HSBC Software Solution
Posted: 6 months ago
Last activity: 6 months ago

Kafka Stream DataFlow consuming Old Data at a Particular time everyday

Hi ,

We are observing an unusual behavior of Kafka stream data set every day at particular time(between 23:00 - 00:00) where it is reading all the records of kafka topic again.

We are connecting to exterenalkafka system and the Kafka configirations are done as below at Pega:

-  configured Kafka Instance to connect to external Kafka server

- configured Kafka stream data set 

- configured Real time data flow with Kafka stream data set as source and with 'only read new records' option

Example scenario: 

- Kafka Topic got 100 records in a particular day and pega reads them as and when they are available in Kafka topic.

- At end of the day(between 23:00 - 00:00) , again reading all those 100 records.

 

Pega Version : 8.1.7

Server : WAS Liberty.

Please provide any inputs .Thanks in advance.

 

 

 

Pega Marketing 8.1 Low-Code App Development Dev/Designer Studio Data Integration Decision Management Conversational Channels Other Industry
Share this page LinkedIn