3 Replies Latest reply on Sep 12, 2019 9:02 AM by Don Wise

    Align refresh schedules

    Ruben Wijnands

      Hi,

       

      I have created a workbook to visulize data originating from our internal database. By making use of Tableau Bridge, I have created a daily incremental extract refresh. However, I am not sure how to set-up the connection to the workbook. The current set-up is visualized below:

       

      How can I keep the data in my workbook fresh? Shall I make a "live" connection, although the underlying data is only refreshed once a day, or shall I time the refresh a few minutes/half an hour after the bridge schedule?

       

      Thank you in advance,

      Ruben

        • 1. Re: Align refresh schedules
          Don Wise

          Hello Ruben,

          You can set whatever schedule works best for your workflow process.  Some folks need a refresh of every 15 or 30 minutes, so they set up an incremental refresh whereas others set up a full refresh once per day.

           

          The agencies that I support will create a workbook that they've downloaded from Tableau Online, so the connection is back to Online.

           

          As Bridge updates (based on your refresh schedule that you set at time of Data Source Publication to Tableau Online) Tableau Online, when they open their workbook (or manually refresh) the data is then refreshed up to the point of the latest full refresh or incremental refresh date/time.  Best, Don

           

          Tableau Online: Keep Data Fresh - Tableau

          • 2. Re: Align refresh schedules
            Ruben Wijnands

            Thank you Don,

             

            However, I am not sure if I fully understand it correctly. I now have created the set-up as follow:

             

            Tableau should retrieve data from our internal database every day, and save it on the (Tableau) cloud. The retrieval is done by a complex query with many calculated fields. I am not sure if Tableau is the right platform to store the query output. One thing I worry about is keeping the history (or recalculate a day when something went wrong). What would you recommend? 

             

            Thanks,

            Ruben

            • 3. Re: Align refresh schedules
              Don Wise

              Hi Ruben,

              Your Incremental and/or Daily (Full) Refresh is done at the Bridge Connection.  You can have multiple schedules for refresh.  So for example, today is Thursday.  You can have a schedule on Thursdays which is incremental every 15 minutes AND/OR also have a full refresh every Thursday.  Just depends on how you want to set it up.  If you're concerned about something going wrong, that's where your initial process and setup is key.  I've fourteen agencies running both a daily incremental and a monthly full refresh with a number of filters and calculations in play.  To date, and it's been running for several years now, we've not had a single issue with having to go back and recalculate a day where something went wrong.

               

              The only time that something can really go wrong is when someone changes the Data Source (tables, joins, etc.) or a Data Source Filter or Extract Filter and then publishes it up to Tableau Online - so initial & follow-up testing on your part or those who are creating authoritative/federated data sources is an absolute must.  I'd recommend a test Data Source and subsequent test folder in Tableau Online where you can sandbox and validate any changes to a Data Source prior to putting it into publication for use by others. That folder would only be viewable by yourself and those who are creating the data sources (via admin/security settings in Tableau Online). Please remember that Tableau is simply reading the data that's available to it.  So if there's a connection issue for some reason and you're not seeing all data with an incremental refresh, that's where a full refresh schedule on top of your incremental refresh schedule could come into play.  If it were truly an issue/concern about history keeping, we'd be seeing many posts on the subject; quite simply, we don't see those on the forums.

               

              Regardless, there have been some posts where folks want to keep a daily 'snapshot' of their data.  Don't recommend that unless you're going to be actively diving into Tableau and adjusting and republishing time filtered datasources daily.  And that type of activity of saving a datasource daily as a new datasource without removing the previous datasources (each their own daily snapshot) would take up memory/storage (leading to more maintenance on your part) in Tableau Online.  I personally haven't found that type of activity to be necessary.  If a datasource is changed however, and is put into production and is problematic, in Tableau Online are a number of Administrative features and functions, of which one of them is to see the revision history of a changed datasource and be able to download it again.

               

              Your graphic isn't incorrect.  The key is that when a workbook is developed in Tableau Online that's when a connection to an available Data Source is made to the user.  There's a dialog box that makes that connection to the data source at time of workbook creation. Thereafter the workbook would be downloaded/saved locally to their machine, where it maintains a connection back to Tableau Online and is refreshed when it is opened or manually refreshed in Menu options.  An example workflow process is below:

              Screen Shot 2019-09-12 at 8.45.49 AM.png

              What we did for our power users who each have their own Desktop/Creator license is to give them a 'Template' workbook as a starting point to their favorite data source.  The template retains the connection back to Tableau Online is updated with fresh data (based on your refresh schedule(s)) whenever they open the workbook or manually refresh through the menu options.  Because they have an existing connection back to Tableau Online with that workbook, if they so desire, they can create a new local workbook and change the data source to any of the other already published data sources (in Tableau Online) and make a new workbook based on the changed/new data source and save as a new workbook thereafter...

               

              Hopefully this explains the process a bit more?  Best, Don

              1 of 1 people found this helpful