Question
2
Replies
78
Views
Deloitte Australia
Posted: November 8, 2018
Last activity: November 11, 2018
Closed
NLP - Stemmer and Lemmatiser
- Do the models built in Pega(like Max Entropy, Naïve Bayes, SVM) support Stemmer and Lemmatisation? If yes, is it inbuilt in Pega or do we have an option choose to have either Stemmer or Lemmatisation for the models.
- SVM and Max Entropy models support Part Of Speech tagging. Is this supported in pega?
- Pega 8 supports summary extraction feature. What is the logic that is used to extract the summary?
Currently, only English language supports Lemming. Reference: https://community.pega.com/knowledgebase/articles/text-analytics-features-pega-7-platform
The ability to configure stemming and lemming is not available to the end business user and is inbuilt in the models we support.
POS tagging is a preliminary step and all models use POS feature sets as input. Again these are hidden from the end user.
Lastly, on text summarization, we have introduced this functionality with 8,1.
You can find all the details at
https://community.pega.com/knowledgebase/articles/extract-meaningful-summaries-text
Summary scores sentences based on the features and predictors. Higher weight is given to topics and facts detected in sentences. Top scored sentences are picked up as the summary.
regards,
Shantanu