4 Replies Latest reply on Jul 9, 2012 4:32 PM by Dan Cory

    Incremental Refresh, how does it work?

    Alan Kwan

      Hi all,

       

      I want to try setting my extract to refresh incrementally, but I am not sure how it works. My data looks like this,

       

      ID         MODIFIED_DATE

      1       06/06/2007 9:28:22 AM

      2       05/17/2012 7:40:32 AM

      3       05/15/2012 9:22:45 PM

      4       05/15/2012 9:22:45 PM

      5       05/15/2012 9:22:42 PM

      6       06/07/2012 9:25:21 PM

      7       05/31/2012 12:03:59 PM

      8       05/31/2012 1:27:57 PM

      9       06/20/2008 9:00:20 AM

          

      It only lets me refresh incrementally by Date (I have many other fields not listed here), ideally I would want to update by Primary Key.  <--Is there a workaround?

       

      If I choose to update incrementally by Modified Date, in a case like ID 4 # 5, would ID 5 not get captured in the update since it has the same modified date as ID 4?

       

      Thanks,

      Alan

        • 1. Re: Incremental Refresh, how does it work?
          Dimitri.B

          My understanding is that incremental refresh only works on dates. I think Tableau needs something continuous to determine what data is new. Other data types may or may not be continuous, but dates always are.

          ID numbers may represent a continuous sequence to you, but Tableau wouldn't know that - they could just be discrete random numbers.

          In your example, if the last item in your data is #4, then incremental refresh will not pick up #5 because it occurred three seconds before #4.

          • 2. Re: Incremental Refresh, how does it work?
            Dan Cory

            Incremental refresh by non-dates is fine - just pick the field that you want use for incremental refresh. The field does have to be an integer type - it's possible Tableau isn't considering your field an integer.

             

            Tableau will take the maximum value of the records it has and ask for all records greater than that maximum to do the incremental refresh.

            1 of 1 people found this helpful
            • 3. Re: Incremental Refresh, how does it work?
              Dimitri.B

              Dan Cory wrote:

               

              Incremental refresh by non-dates is fine - just pick the field that you want use for incremental refresh. The field does have to be an integer type - it's possible Tableau isn't considering your field an integer.

               

              Tableau will take the maximum value of the records it has and ask for all records greater than that maximum to do the incremental refresh.

              I mocked up some data to test this, and I couldn't see how this would work. Only dates were available to base incremental extracts on. Am I missing something?

              The Tableau help does say: "Alternatively, you can use an ID column that increases as rows are added to the database", but how?

               

              incremental extract field options.png

              • 4. Re: Incremental Refresh, how does it work?
                Dan Cory

                Dimitri -

                 

                Sorry you are still having trouble with this. I can't say for sure what the problem is without you posting your workbook.

                 

                One possible problem is that your data comes from Excel. Numeric columns in Excel are always doubles, so they can't be used for incremental refresh.

                 

                Dan