When I restart the ProcessEvents dataflow, we can successfully read the events from external kafka. We’ve configured the read options for the dataset as ‘Read existing and new records’, which means, when we restart, its reading the existing events and the same we could trace. Just want to understand, since how many days old the existing events gets picked up? Because, there were more than 2000+ events existing in Kafka and only 750+ events processed after restart.
@kaman , thanks for the response. So, the 2000 records are in a span of a month. and we could see only 750+ records got processed. What do you think the cause for not processing the whole records? as I mentioned, the 750+ records are in a span of 2 weeks.