2 Replies Latest reply on Mar 29, 2016 6:47 AM by Patrick Alpers

    Remove extract data

    Patrick Alpers

      I have an extract of loan data that contains month end snapshots as well as the most recent processing day (last business day). So, today I'll have data for 1/31/2016, 2/29/2016, and 3/8/2016. Tomorrow I'll have data for 1/31/2016, 2/29/2016, and 3/9/2016. On April 5th I'll have data for 1/31/2016, 2/29/2016, 3/31/2016, and 4/4/2016. The month end data never changes, but I only want the most recent day during the current month. I have a MS SQL table and stored procedure that fills the table this way each day (deletes anything other than month end data, then adds last business day). I've been doing full extract refreshes every day, but every month it's taking longer and longer. I've looked into incremental extracts, but I can't find anything built in to Tableau that would get rid of the non-month end data.

       

      I found this article: Refreshing Large Extracts Faster which sounds like a plausible solution, but I'm looking for something simpler if possible.

        • 1. Re: Remove extract data
          Jeff Strauss

          I managed to implement the solution that is in the link that you referenced.  Though it is a manual solution that we deal with right now on a quarterly basis.  We call it "back dated incrementals (which updates the previous 90 days), but it could be used to get rid of older data too.  In any case, we have a solution on the shelf to have a fully automated solution and we will probably get back to finishing up the build of this solution within a few months.  But to answer your question, out of the box there's full or incremental.  And then with some clever scripting and a bit of added complexity, you can do anything.

          • 2. Re: Remove extract data
            Patrick Alpers

            Thanks Jeffery. If you get an automated solution going, I'd be interested in what you worked out.