13 Replies Latest reply on Feb 10, 2019 5:25 PM by Mark Wu

Scaling Tableau - Set Extract Priority Based on Duration

Mark Wu Dec 5, 2016 8:41 PM

Are you facing the situation that your Tableau server backgrounder jobs have much longer delay during peak hours ?

I managed to reduce 50% extract waiting time after increased priority for all extracts with duration below median runtime. Read my recent blog @ Scaling Tableau (2/10) - Set Extract Priority Based on Duration - Silicon Valley Enterprise Tableau User Group

Please share your tips on different ways to increase extract efficiency.

1. Re: Scaling Tableau - Set Extract Priority Based on Duration

Nick Roush Dec 6, 2016 6:35 AM (in response to Mark Wu)

This is awesome Mark! I have been manually prioritizing based on time and it is a huge pain when you have thousands of extracts to update.
Like Show 0 Likes(0)

Actions
2. Re: Scaling Tableau - Set Extract Priority Based on Duration

Jeff Strauss Dec 6, 2016 7:13 AM (in response to Mark Wu)

thanks Mark! I never knew of the sorting feature, is it new?
Like Show 0 Likes(0)

Actions
3. Re: Scaling Tableau - Set Extract Priority Based on Duration

Obed Tsimi Jun 29, 2017 7:17 AM (in response to Mark Wu)

Thanks much Mark Wu ,

Very helpful.

Just a few questions for you.

How many backgrounders do you have and what version of Tableau server are you running?
Like Show 0 Likes(0)

Actions
4. Re: Scaling Tableau - Set Extract Priority Based on Duration

Andrew Macey Dec 6, 2016 4:24 PM (in response to Mark Wu)

I still find a backgrounder utilization viz very useful for identifying offending sites Re: Viewing Tableau 9 Background Tasks Queued Getting users with long running jobs to use a Serial rather than Parallel schedule also helps to stop them hogging all the backgrounders.
Like Show 0 Likes(0)

Actions
5. Re: Scaling Tableau - Set Extract Priority Based on Duration

Mark Wu Dec 6, 2016 7:35 PM (in response to Andrew Macey)

I am on v10.0 with about 3,000 extracts per day and 40 backgrounders. tabadmin set backgrounder.sort_jobs_by_run_time_history_observable_hours must be a v10.0 new feature. I have not seen this set option at v9.2 admin guide or v9.3 admin guide but it shows the 1st time on v10 admin guide. I have not seen Tableau advertises this feature anywhere in official v10 new feature list. I think that this is another great big brother v10 new feature similar as Desktop license reporting. One more good reason to upgrade to v10 if you have not done so yet.
1 of 1 people found this helpful
Like Show 1 Likes(1)

Actions
6. Re: Scaling Tableau - Set Extract Priority Based on Duration

Mark Wu Jun 30, 2017 9:15 AM (in response to Obed Tsimi)

60 backgrounders on V10.2 now with about 6,000 extracts/day
Like Show 1 Likes(1)

Actions
7. Re: Scaling Tableau - Set Extract Priority Based on Duration

Mark Wu Oct 19, 2018 9:43 AM (in response to Mark Wu)

80 backgrounders on V10.5 handles 15,000 extracts/day with avg 10 min delay at peak hour
Like Show 0 Likes(0)

Actions
8. Re: Scaling Tableau - Set Extract Priority Based on Duration

Mirko Langhorst Oct 24, 2018 9:17 PM (in response to Mark Wu)

Where can I find the technically implementation of the script to set this automatically?
Like Show 0 Likes(0)

Actions
9. Re: Scaling Tableau - Set Extract Priority Based on Duration

Mark Wu Oct 30, 2018 8:30 PM (in response to Mirko Langhorst)

The key trick is to update tasks.priority (tasks is Postgre table name, priority is the column name) to whatever priority you wanted.
Like Show 0 Likes(0)

Actions
10. Re: Scaling Tableau - Set Extract Priority Based on Duration

Mirko Langhorst Nov 12, 2018 11:54 AM (in response to Mark Wu)

Hi,

has anybody ever checked as well the built-in heuristic feature of Tableau? --> the default config YML file has some hidden parameters which can be set?

# Some average run time heuristic for different background jobs, used when calculating background job's jobRank
# the heuristic numbers below are based on background job data on 10ay before 08/08/2017 9:00:06
backgrounder.refresh_extracts.heuristic_average_run_time: 250
backgrounder.increment_extracts.heuristic_average_run_time: 124
backgrounder.single_subscription_notify.heuristic_average_run_time: 27

# Days that objects are allowed to be in the datasource_events table before removing them.
backgrounder.datasource_events_retention_days: 14

#Controls: whether to calculate background job rank or not when queueing a job
#And whether to update task run time when recording a job as finish
backgrounder.enable_task_run_time_and_job_rank: false

sorry, just read the full article from Mark: Scaling Tableau (2/10) - Set Extract Priority Based on Duration - Silicon Valley Enterprise Tableau User Group --> awesome explanation!
Like Show 0 Likes(0)

Actions
11. Re: Scaling Tableau - Set Extract Priority Based on Duration

Iwona Wiktorowska Nov 13, 2018 1:32 AM (in response to Mirko Langhorst)

Also interested in the code as I was planning on writing something similar in python.
Like Show 0 Likes(0)

Actions
12. Re: Scaling Tableau - Set Extract Priority Based on Duration

Rafael Schaffer Jan 29, 2019 10:24 AM (in response to Mark Wu)

How many CPU's for the 80 backgrounders?
Like Show 0 Likes(0)

Actions
13. Re: Scaling Tableau - Set Extract Priority Based on Duration

Mark Wu Feb 10, 2019 5:25 PM (in response to Rafael Schaffer)

We had 96 CPU for 80 backgrounders. For 96 CPU, you can actually config more than 80 cores. It depends on avg and peak time CPU% tolerance level.
Like Show 0 Likes(0)

Actions