Posted: 20 Sep 2019 7:04 EDT Last activity: 7 Oct 2019 1:45 EDT
Unable to combine two data sets in data flow
We have a requirement where in bulk data from multiple data source will come. We have tried to create a data set uploading a data from csv file. When i tried to combine 2 different data sets using compose for the second data set i am getting error "
This DataSet cannot be used as a secondary Source as it cannot be browsed by keys
I have defined primary key as ID for both first and second data set . I am unable to figure out the way to fix it. Any inputs or help is appreciable.
The CSV dataset indeed doesnt support that, what it really means that there is no way of indexing the records in such a dataset quickly. The workaround is to copy the CSV dataset into something that can be used for a join/merge, most commonly a DDS (Cassandra) dataset. So you first run a dataflow that copies the entire CSV into a DDS set (with appropriate indices defined), then join/merge that one in your main flow.