the best way to see what happens is to try it and watch the perfmon metrics. I don't think that Tableau software will constrain you, but it's more directly related to how many parallel backgrounders can run prior to your cpu or memory maxing out.
shawn harvick wrote:
- In dealing with a worker that is dedicated strictly to background tasks, has anyone pushed a worker to N/1?
- Is it possible / efficient to put 8 backgrounders on a 6 core machine?
1. I haven't.
2. No idea, try it and let us know If possible I'm pretty sure it wouldn't be efficient. I wonder if Server has built-in checking to make sure the processes don't outnumber the cores?
Why do you ask?
We are in the midst of trying out an additional 8 cores bringing us to 16 cores. Part of this is trying different configurations over a 45 day period to see what works best and what our options are. We are extract and view heavy with a need for HA.
Thanks Jeffrey! I may stick with 6 cores 6 backgrounders and see how it goes. My gut tells me that exceeding the number in cores in number of backgrounders won't be pretty.
ok, let us know how it goes.
There's a lot of "it depends" in there. For example it depends on how much Refresh activity is configured on the system, and how "spaced out" the Scheduled Tasks are. If you have a fairly small number of Tasks evenly distributed throughout the day, you'd need fewer Backgrounders versus if you have a mountain of Tasks all firing off at once.
Certainly if you can dedicate a Worker to just Backgrounders and you have the licensing capacity, then adding more Backgrounders in theory can't hurt. But there should certainly be some sort of threshold at which more Backgrounders either stops helping, or consumes system resources that don't need to get consumed.
I have tried this , I have 4 cores and configured 6 BG services on this worker ,It ran fine sometime but sometimes in morning I got failure on extract refreshes. .now again go back to 4 BG services .
Below is the older configuration which caused extract failure issue on mornings . (its like twice in a week )
And last night only we have changed it to 4 BG services like below
Per my experience N/1 is not efficient.
1 of 1 people found this helpful
Hi Sunil -- not sure I 100% agree, as, again, there is a lot of "it depends". I run a fairly massive environment (hundreds of Scheduled Refresh tasks) with just 3 backgrounders. There are trade-offs that one has to make. More is not automatically better, and I am very surprised by the statement that N/1 is not sufficient, because that is certainly not what Tableau itself says about the matter.
2 of 2 people found this helpful
Here at Tableau we are currently running 10 background tasks on each of our two dedicated 16-core backgrounder hosts. Primarily, increasing past 8 was intended to reducing subscription delays--subscriptions don't use up nearly as much CPU as refreshing extracts does, and we needed more throughput for the 6AM schedule, which sends hundreds of subscriptions out every day. We regularly max out our backgrounders with extract refreshes alone, and we have not seen failures due to lack of CPU resources. These are pretty beefy Dell 620s though, with 96 GB RAM, and SSDs for storage.
Thanks Matt. Do you think that the optimal number of backgrounders is somewhat context-sensitive (like you mention the 6:00 AM schedule), or do you prefer to go to a hard equation like N /1 or ((3 *N( / 2 * Cores)) or suchlike?
For us at least, if we have a host dedicated to nothing but backgrounders, the only real risk of overloading it seems to be a reduction in efficiency. So we've been relatively unafraid to throw some punishment on those boxes in our various Server implementations, and it hasn't bitten us so far. The one minor annoyance is that if you increase past 8, you get the nags whenever you save the config that tell you you're not supposed to do it.
But I haven't done any testing on how efficient extracts are when you've overloaded the box with them, versus keeping with the recommended number of Backgrounders per core. So I can't say what you might suffer in terms of increased delay, if you're saturating your box with parallel extract refreshes. To me, it seems like a low-risk test to increase it, and see what it does to refresh durations. But there is definitely a benefit if you're running multiple Subscriptions schedules at the same time, and need them to complete more quickly.
So to answer your question, no, I don't like to go with a hard equation. Primary factor is "did you isolate Backgrounders to one host responsible for nothing else?" If not, I'd be very hesitant to put much work to them during business hours when users are active. If so, you have some freedom to play around.
Okay thanks. It's also fair to say that you might not have Licensing limitations like we customers have.
It is very fair to say so, actually, and while I am very proud of how well we "dogfood" our own products, I am also painfully aware that we do not do so in that area--but would very much like us to start.
1 of 1 people found this helpful
No worries -- was not meant as a criticism (after all, I get a fairly nice discount on networking and communications equipment where I am ) .
I only brought it up because one of the other factors in deciding how many instances of each Tableau Server process to configure involves having a finite bound imposed by the License. No free lunch. So going overboard on one process type might starve another. That's all.