The solution depends on getting the data that is actually needed to be worked on.

For example in this case instead of getting all the flight data for all the passengers for 2 years down to the client for a crew member to access in offline, we need to refine the filter criteria at point of retrieval on server during the packaging sync request::

Suggestions - 

1. Add a new column for the crew member to the flight master data table (if possible) so the sync gets filtered data only for the flight manifests of which the crew member is part of. It may not be possible to change existing tables / schema. Instead we can do a Join on the crew member roster with the flight passenger data .

OR

2. Add the  condition to the report definition to essentially get only data for  the current day (or maybe day before and day ahead) so that crew member gets immediately relevant data .  300 * 200 = 60000 rows.

Once we get that manageable subset we may use large data pages to get data for a particular flight when we run the case.

Thoughts ?

Hi, 

Did you tried using large data page for this particular requirement ? Check out the below link for the same,

https://pdn.pega.com/tutorial-how-use-large-data-pages-store-large-refe…

Create a new datapage, and source it from an Activity. In the activity, load data from the original datapage (the huge one) and filter it whatever way you like (e.g. in a Java step). No need to pass any parameters = no problems.

Use that new datapage as a source for your UI controls (lists, forms, etc.). Behind the scenes it will make use of the huge one and filter it.

Sounds like this should work : would this require us to create one datapage for every unique set of params though ?

So if we had (say) two parameters 'name' and 'city' : and one big datapage on the backend which associates names with cities ; and we want to fetch only the rows where 'name=joe' and city='new york'; would we need a datapage for this combination : something like D_MyPage_joe_new_york ?

EDIT: Or does the OOTB 'sync' process already carry a unique identified for the device of some sort  ? If there is such a 'device_id' - we could (with some customization) use that as a 'namespace' for unique params.

So I mean: lets say our device has an ID of 'device123': and this particular device is interested in only the subset of data where 'name=joe, city=new york' - but we are unable to pass the parameters directly in the sync request; so *maybe* we could send up another request to PRPC -uploading our parameters 'name=joe, city=new york' to *another* datapage which indexed by our 'deviceID' 'device123' ; *before* the sync is made.

That way our filtering Activity (presuming this also has some visibility of such a 'deviceid') could look up the parameters that way.

But there is a lot of hypotheticals in that guess above.....