I am developing WDC. It will speak to REST endpoints and the response I have to publish to tableau datasource.
Rest endpoint will connect to cassandra based on date-time, and returns millions of records. These records I have to send to tableau datasource via WDC. And I have to refresh the datasource everyday by appending the current day's data with the existing data in the datasource.
I tried with 1000 records it is working perfectly fine. But if my json response is more than a million of objects, how to handle this in WDC. And how to append the only previous day data without reloading the entire records via wdc.