Question
2
Replies
103
Views
Plexus Tech
Posted: August 28, 2019
Last activity: August 29, 2019
Closed
Partition keys in Stream Data Set rules
Hi,
I created a Data Set to get messages from a kafka queue that has 2 partitions.
How do I assign value to the property specified as Partition Key?
Attached captures with the Data Set definition.
Thanks.
Moderation Team has archived post,
Close popover
This thread is closed to future replies. Content and links will no longer be updated. If you have the same/similar Question, please write a new Question.
Posted: 1 year ago
Hi Miguel Calderon,
What is the pega version that you are using?
From pega 7.4 ,by default number of partitions for kafka and stream dataset are 20(no configuration is needed). If you want to change this default number of partitions, go to kafka folder (in application server folder) and open connect-distributed.properties file and change the value of num.partitions.
By configuring partitioning you can ensure that related records are sent to the same partition. If no partition keys are set, the Kafka data set randomly assigns records to partitions.
Please go through below link for more information related to kafka data set creation.
https://community.pega.com/sites/default/files/help_v74/procomhelpmain.htm#rule-/rule-decision-/rule-decision-dataset/dsm-creating-kafka-data-set-tsk.htm
Generally to set the value of the property which is used as partition key for the dataset, you need to send the value in request itself.
Let's say i am using Age property as my partition key for stream data set, I need to send Age property value along with the required properties while sending the request to stream data set.
Hope it helps.Let me know if you have any queries.