Sandeep Punugu (svpunugu)

svpunugu Member since 2009 3 posts
Posted: October 2, 2020
Last activity: October 2, 2020
Posted: 2 Oct 2020 8:52 EDT
Last activity: 2 Oct 2020 11:40 EDT

Kafka Consumer Configuration in Pega 8.2.1

We have created a data flow rule for real time processing connecting to a topic(has 4 partitions) via the kafka dataset(json). We are able to send and receive responses. Since most of the configuration is under the hood, we are not sure how the consumers are configured within pega and if we can make any changes to the configuration. Below are the questions. The PRPC version is 8.2.1

1.    Can we create consumer groups for Kafka in pega? If yes, how do we do it. 2.    How are consumers created in pega? Are consumers created automatically depending on the number of partitions on a topic? Does a consumer co-relate to a thread on the data flow rule 3.    How can we change the partition assignment strategy ( I believe default is Range Assignor) 4.    How are the partitions assigned across the consumers? 5.    How does pega ensure “Exactly One Semantics” when consuming messages

Thanks, Sandeep  

Pega Platform 8.2.1 Decision Management Healthcare and Life Sciences Senior System Architect