Posted: 5 Jan 2021 9:56 EST Last activity: 6 Jan 2021 6:58 EST
Does Materialized IH summary dataset fetch data from relational database or Cassandra?
I have an summarised dataset for aggregating data. Its base dataset is pxInteractionHistory. I have materialized it and to confirm that it fetches data from Cassandra, I stopped the DDS service and executed the dataset. I had expected that it will not be able to return results and throw error. But it returned results from relational database. Can anyone please explain the behavior?
I had gone through the article already. But what I wanted to know is why was the data in the dataset fetched from relational database when I had switched the summarised dataset to materialized mode? I had turned the DDS services off so ideally, it should have thrown error or returned no results.
Can you please explain this behavior?
Posted: 1 year ago
Updated: 1 year ago
Posted: 6 Jan 2021 5:12 EST Updated: 6 Jan 2021 5:13 EST
Let me explain why this behavior happened. In Pega, we try to always ensure data availability even when DDS service is not available. In order to do so, whenever a read operation happens for a materialized summary dataset, the following behavior occurs:
We read the materialized aggregate and its timestamp and the timestamp of the last customer interaction. This happens at the customer's level:
If the timestamp of the last interaction is later than the timestamp of the aggregate (which means there are more data that needs to be aggregated) we go to the relational DB to read the Interaction history data and read all the interactions between these two timestamps and add them to the materialized aggregate and return the result to the customer. Otherwise, we return the materialized aggregate directly.
If the Cassandra read operation fails, then we read all the Interaction History records from the relational DB for the given time window and return the aggregated result to the customer. Obviously, in your case this is what happened.
Hope that helps, please let me know if you have further questions!