8 Replies Latest reply on Jun 7, 2018 4:46 PM by Gil kim Branched to a new discussion.

    Question about using REST API to kick off an extract refresh

    Katherine Woods

      Hi all,

        We have a situation where we are using workflows to update our data. The workflow completion times are unpredictable and are making it difficult to find the right schedule for our extract refreshes. So, we thought to put a REST API command to refresh the associated extract(s) at the end of each successfully completed workflow. After researching the REST API documentation,

       

      Here is the info listed in the reference:

      Run Extract Refresh Task

      Runs the specified extract refresh task.

      This method runs the scheduled task for the data source extract or the published workbook that connects to the data extract. You must first schedule the task for the extract refresh. You can do this using the Create Schedule method to create the task. To get information about the extract refresh task, use the Get Extract Refresh Tasks method, which returns the extractRefresh ID that you use as the task-id.

      The method adds the refresh task to the backgrounder queue, at the default priority. Depending upon the backgrounder load, the task might not run immediately. The method returns information about the backgrounder job responsible for running the extract refresh task, including a job id that can be used with the Query Job method to query the status of the extract refresh.

       

      my question to this group is as follows:

       

        If the REST API can only run an already scheduled task by providing the site-id and task-id as parameters to the request, then wouldn't the extract refresh run twice? Once when kicked off via the REST API at the end of our workflow and again by the Tableau Server scheduling process?  Must we use REST AP (and not the server scheduler) to create the scheduled task, get the task id, run the task id, then delete the scheduled task for each workflow run?  Is this correct?

       

      Thanks for the feedback!

        • 1. Re: Question about using REST API to kick off an extract refresh
          Matt Coles

          Create a schedule named "ad hoc refreshes" or something to that effect. Then, either set the schedule to run rarely, at a time that it running will not matter, or disabled it entirely. I'm not certain how the REST API handles disabled schedules, but we use tabcmd regularly to force a schedule of extracts to refresh, when our data transformations are completed. So my expectation would be that the REST API would work the same way.

           

          Another alternative is to simply use tabcmd refreshextract, which does not require a schedule (come to think of it, I think I've always used it against items that already had schedules...but I still think it should work for what you need).

          • 2. Re: Question about using REST API to kick off an extract refresh
            Katherine Woods

            What would be the benefit of using tabcmd over REST API to refresh extracts, or visa-versa? I have used tabcmd in the past, but that was using a script in a single Tableau Server environment. We assumed that using REST API added an additional level of access and security when different environments were involved (data environment / workflow environment/ Tableau Server environment) which is why we are researching this method.

             

            I think that you are saying that we can create a schedule on the server with no tasks, then we need to use the REST API to add the task to the schedule, get the scheduled task id, execute the refresh of that task id, then remove the task from the schedule each time the workflow completes, should we go with the REST API option.

             

            Thanks for your response,

             

            Katherine Woods  |  Senior Software Engineer, Global Services Managed Analytics|

            • 3. Re: Question about using REST API to kick off an extract refresh
              Matt Coles

              The REST API might be a bit faster to execute, if you're going to be calling this a lot--but I doubt you'd need to call it frequently enough to matter (multiple times a second, constantly). You'll have to build in a way to pass the right credentials to Server to log in using either method. For TabCmd you'll need to use the --no-cookie option so that each session remains independent of the others, if you ever talk to two Tableau environments at the same time.

               

              Actually I was suggesting you assign the schedule manually to the item you want to refresh, then use the REST API (or Tabcmd) to make the calls necessary to refresh it. But your way would work too, I think.

              • 4. Re: Question about using REST API to kick off an extract refresh
                Katherine Woods

                Matt,

                  In looking at the postgres database, it looks like the server assigns a new task ID every time a scheduled task is executed.

                 

                 

                Given that, my understanding of the Rest API is even more fuzzy. From what I can see in the commands listed, I can create a schedule and run a task, but I can’t find any command to actually schedule a task (extract refresh) to a schedule so that I can then run it.

                 

                 

                To run extract refresh task, one parameter is the task id. I can create a schedule using create schedule, but how do I assign a task to that schedule using REST API? Using GET Extract Refresh Task only returns tasks assigned to the schedule, which should be none if I just created the schedule. Where is the missing command to create the task in the schedule?

                 

                The info in the heading for Run Extract Refresh Task implies that Create Schedule also schedules the task, but that doesn’t look like the case when you look at the actual examples listed:

                This method runs the scheduled task for the data source extract or the published workbook that connects to the data extract. You must first schedule the task for the extract refresh. You can do this using the Create Schedule<https://onlinehelp.tableau.com/current/api/rest_api/en-us/REST/rest_api_ref.htm#Create_Schedule> method to create the task. To get information about the extract refresh task, use the Get Extract Refresh Tasks<https://onlinehelp.tableau.com/current/api/rest_api/en-us/REST/rest_api_ref.htm#Get_Extract_Refresh_Tasks> method, which returns the extractRefresh ID that you use as the task-id.

                 

                 

                Thanks,

                Katherine Woods  |  Senior Software Engineer, Global Services Managed Analytics|

                • 5. Re: Question about using REST API to kick off an extract refresh
                  Glen Robinson

                  Hi Katherine

                  If you are on Tableau 10.5, you may want to look at the Update Workbook Now (or Update Datasource Now)

                  https://onlinehelp.tableau.com/current/api/rest_api/en-us/help.htm#REST/rest_api_ref.htm#Update_Workbook_Now%3FTocPath%3…

                  This negates the need to have Schedules, and Tasks, etc

                  Hope this helps

                  Glen

                  1 of 1 people found this helpful
                  • 6. Re: Question about using REST API to kick off an extract refresh
                    Jeffrey Line

                    Hi Katherine,

                     

                    We don't really have many scheduled refreshes within our Tableau server environment, but rather use tabcmds to initiate the data extracts. We can trigger either a data source or a workbook to be refreshed. We've added a job at the end of our data warehouse processing to trigger the refreshes once the data has been updated in the source for tableu.

                    We choose to use the tabcmd since it was the only option at the time to trigger the refreshes. To me either one would work to trigger what you want.

                     

                    Hope this helps.

                     

                    Thanks

                     

                    Jeff

                    • 7. Re: Question about using REST API to kick off an extract refresh
                      Katherine Woods

                      Thank you for this information, we have used  tabcmd for this in the past, but are wishing to try REST API as the ability to kick this off from the end of a workflow in another environment would eliminate a polling or other communication process to signal the correct end of the workflows, greatly simplifying the process, if we can get it to work.

                       

                      Katherine Woods  |  Senior Software Engineer, Global Services Managed Analytics|

                      Desk 512-865-3246|  Mobile 512 791 8226 | kawoods@visa.com<mailto:kawoods@visa.com>

                       

                       

                      • 8. Re: Question about using REST API to kick off an extract refresh
                        Gil kim

                        Hello Katherine,

                         

                        I'm currently dealing with this as well. I have a script that will kick off an extract (this is triggered by a SQL query that will only return if the workflow has ended), check the postgresql database to make sure the extract was successful, update a schedule to the next 15 minute slot, and then that schedule will run. I also have another script that runs mid month and will will update the newly modified schedule to a month out so that our schedule never kicks off without the correct data in the tables. I was surprised to see that you're looking for the exact thing I'm looking for: a way to add subscriptions to a schedule. I cannot for the life of me find any way to add anything to a schedule. If this was possible, I would create a new schedule, add my subscription, kick it off and then delete. Also, if I could disable my subscription, that would be great.