Posted: 7 May 2018 1:36 EDT Last activity: 18 May 2018 8:42 EDT
How can I measure the performance of Predictive models?
When it is a question of how I measure performance of a predictive model used in a strategy , one of some suggestions Pega suggest to use another adaptive model to monitor the performance of a predictive model.
is there any sample strategy diagram or document available to understand this pattern in details?
You can feed the output of your predictive model (for example the 'probability to churn') to an adaptive model as a predictor. The performance of the predictor (AUC) will then be visible in the ADM monitoring.
Please note that you would still need to capture responses (actual outcomes) to obtain an AUC.
In the example below the 'churn rate' output of a predictive model is mapped to a 'ChurnRate' property in the Output Mapping tab.
The adaptive model is then passed the value of 'ChurnRate' as a parameterized predictor.
P.S. note that the adaptive model component is in the execution path
in the sample strategy, if I have to use 2 predictive model for comparing performance between those 2 predictive model, then if I have to use champion challenger between those 2 predictive model then how the model will look like?
For completeness sake: If you want to switch between different types or versions of an adaptive model, based on SR properties like e.g. evidence or performance, you would typically use a group-by component, not a switch or champion/challenger