did you tried to create an incremental refresh instead of a full one?
by the way you can schedule an incremental refresh during the weekday and a full refresh (for performance) during the weekend
Unfortunately the incremental refresh won't work for this situation. The fields in the claim records can be updated with new values days, weeks, or even months after they are initially created. The extract would not pick up the updates on a given claim because it has already seen that claim in an earlier refresh.
Basically, unless the underlying database table is modeled so that records are only added (never updated or deleted) then an incremental refresh should not be used. Your extract would quickly get out of sync with the underlying data source because the refresh would never delete or update records.
2 of 2 people found this helpful
you can use the Tableau Data Extract API (available in Phyton/C/C++/Java for WIndows and Linux) to create "programmatically" your Tableau Data Extract.
Using this API you can append new records to an existing data extra. So, for example, you can freeze a file extract with the data of the previous 5 years and, every day, create a new extract appending only the current year data.
More info are available here: http://onlinehelp.tableau.com/current/pro/online/en-us/help.html#extracting_TDE_API.html
@Paul Merrill, I have the same problem to address. Did this solution from Pietro work for you?
I have the same problem as well. My extract takes 6 hours to incrementally refresh and was thinking of dividing the data into months. I haven't found a way to do a union extracts. I hope someone from Tableau has a suggestion in addition to Pietro Luigi Gentini's.
probably not the silliest question to ask - yet do you have a workbook to share that you are using to display data for the end user? surely they are not using your database to search for information - if that is the case you should be doing that in the database or with a front-end. also are there more than 1 users using this dashboard?
are you just looking for the current or last year data - if so you could put a data source filter in to make it more refined for your needs. would that suit you better?
using a data source filter you could also filter it by region or team.