Question
Difference in count of offers initiated in distribution test and real campaign run
I have a campaign where in the strategy I am using 1 offer ( eg: ABC) and I am checking in IH if this particular offer has been accepted by any customer in the segment. If any customer in the segment have received this offer ABC in last 5 days then that particular customer will not receive the offer anymore.
For this I ran distribution test before running the actual campaign. I got 11k offer initiated in distribution test.
After 2 hours when I ran the real campaign, count of offer initiated is 15K.
I do not understand why there is so much difference in the count of offer initiated in distribution test and in real campaign.
Can anyone please explain why there is so much difference in the number of offer initiated in distribution test and in real campaign run?
@Sriram Krishnan
Thanks for your reply
I am not refreshing the segment at the time of campaign run. It is the same segment which I had used for distribution test.
I am checking in strategy itself for 5 days using IH component and filter component. I am not checking in segment.
Then to do analysis, I took few customers who were not targeted during distribution test and tried to check IH for them. I found that these customer did not received the offer (eg ABC) in last 5 days. Not sure why these customers were not targeted in distribution test.
Thanks,
Taniya
Hi @TaniyaD3
The very same happened to me few times. In my case, there were few checked out rules prior to running the distribution test. Dist. test is running against your rule set whereas campaign execution (the underlying agent) is running only the checked-in rules. That's why there was a delta between the actual campaign run counts and distribution test outputs. After checking in all the rules, I was able to see the same counts both in campaign and dist. run. Please make sure that both campaign and dist. test both are executing the same rule sets (version).
Regards,
@TaniyaD3
Are you re-running the segment at the start of the distribution test and / or real campaign ? There is a configuration in the campaign or distribution test to automatically do this.
My current guess is this behavior is because of two possibilities:
1. You are re-running the segment only when the actual campaign run happens, because of this your distribution test had old data in the segment and hence the difference in audience size. OR 2. The check what you are doing in the segment for "if in last 5 days" - also has a time component to the IH query, so in 2 hours, the # of people who got eligible becomes higher.
If you think both the possibilities are not there, then please create a Pega support request to look into the scenarios much more in detail.