Hi,

When the underlying data changes, you can refresh the extract by selecting a data source on the Data menu and then selecting Extract > Refresh. Extracts can be configured to be fully refreshed, replacing all of the data with what’s in the underlying data source, or incrementally refreshed, adding just the new rows since the last refresh.

Full Refresh

By default, extracts are fully refreshed. That means that every time you refresh the extract, all of the rows are replaced with the data in the underlying data source. While this kind of refresh ensures you have an exact copy of what is in the underlying data source, it can sometimes take a long time and be expensive on the database depending on how big the extract is.

If the extract is not set up for incremental extract, selecting to refresh the extract will fully refresh the extract. If you’re publishing the data source to Tableau Server, you can specify the type of refresh in the Scheduling & Passwords dialog box. Most data sources support an incremental refresh.

Incremental Refresh

Rather than refreshing the entire extract, you can set it up to only add the rows that are new since the last time you extracted data. For example, you may have a data source that is updated daily with new sales transactions. Rather than rebuild the entire extract each day, you can just add the new transactions that occurred that day. Then once a week you may want to do a full refresh just to be sure you have the most up to date data.

Mahfooj

I have only three question to ask to decide whether to go for Full Refresh / Incremental Refresh. If all the answer is "Yes" to the following questions then you should go for Incremental refresh, else for all condition you have to approach for Full Refresh.

A. Does your data always gets inserted and never gets updated / Deleted ?

B. Does your data set have a unique key (A integer serial number which always increase or a date that always increase)?

C. Does your relationship of the dimensional data will not change in future ?

Howdy,

To add onto some of these responses, it is good practice to use full and incremental refreshes. We will often use incremental refreshes during the week/working day and then a full refresh on weekends or in the evenings after working hours depending on the client need. We do this for speed purposes. When using an incremental refresh the data is saved elsewhere on the hard drive. You can defrag the the disk to avoid some of the slow down over time but its best practice to full refresh at least once a month.

Cheers!

Carl Slifer

InterWorks

Hey everyone!

I have some database tables that we have been having an issue with updating. I created some visuals based on the information schema.

the data that i have is laid out as Table_Name, Table_Row (Number of Records), Create_time(the time the table was updated/created).

Our tables in this database are dropped nightly and then added in so the Create date/time is usually always less than 24 hrs behind. I'm wondering if a row comes in that says

     TABLE NAME     RECORD COUNT     CREATE

     table_1                  231224                    11/1/2016

If I set this up to do an incremental refresh and it pulls the next day as

     TABLE NAME     RECORD COUNT     CREATE

     table_1                  241224                    11/2/2016

is Tableau going to come up with this view?:

     TABLE NAME     RECORD COUNT     CREATE

     table_1                  231224                    11/1/2016

     table_1                  241224                    11/2/2016

OR would it just overwrite the same row?

Dear Kyle - If your source is truncated or dropped on a periodic basis, then I should strongly suggest you to go for Full Refresh. Otherwise there is a high possibility of data duplication in Tableau.

If you still want to implement incremental refresh, you can do that. In that case you need to very strictly maintain a key (A numerical integer) that will ever increase. Use that key to derive unique in the Table.

If you can't handle this within Tableau for some reason, then you maintain the table in a database staging and Tableau should read from that Staging table, not from the original table.

Thanks,

Manideep

Thanks Manideep Bhattacharyya

Hello Kyle - If this helps, then could you please mark this as "Correct Answer" to close this question thread. - Manideep

I don't think that I can do that since I'm not the creator.

Hi Carl,

You mention that you perform the incremental refresh regularly and the full refresh once/week.  When using Tableau Server, is there a way to manage this with a schedule or is your team performing this function manually?

Hi Amber,

All of this is controlled via Tableau Server. Nothing stops you from setting multilevel schedules for datasource. You can ha e one incremental daily and other ad a full extract weekly.

Hi Carl, can you elaborate on that please? I intend to run an incremental refresh on weekdays, and a full refresh on month-end. There is another wrinkle to this where I need these schedules tied to Autosys jobs, but that's not my question. Can you provide some details on how to set up multi-level schedules? Using tabcmd?

Hi Manideep, can you clarify/add detail around item #3? Thank you!

Hi Ambarish,

do you have any updates as to how we can do this?

I have a similar situation where i have to do a full refresh on weekend and incremental on Weekday.

Any pointers will surely help.

Other experts please advice.Amy SchneiderJeff SolomonCarl Brenner