6 Replies Latest reply on Oct 30, 2016 10:58 AM by John Abdo

    CPU usage by project/department  - chargeback criteria

    Hitesh Sharma

      Hi.

       

      We are new to Tableau Server. Multiple business departments would use the same environment for different purposes. Each department would have multiple projects.

      We are already working on the storage capacity aspect by monitoring project sizes and establishing a funding model for additional storage capacity as needed. No problem there.

      We have started with 2 workers but we would go out of CPU capacity one day. We would be asked which department is using what proportion of the capacity so that addition of capacity can be funded.

      We may or may not chargeback but usage data by departments is something that people would be interested in.

       

      To my understanding so far, we can track who accessed what view when. I believe the duration to process the action is captured. When was some extract refreshed and how long it took. However, there are no stats in postgres DB on what system resources did an action or extract take.

       

      How are established Tableau server administrators managing this aspect?

       

      - Is it just that we identify the time windows when CPU is running short (after other attempts of optimizing usage) and see who all are active in that window and approach with that data?

      - Can it be assumed that each action occupies one CPU core for it's duration (Vizql, data engine, data server working together). Certainly not accurate, but can it give a good estimate of CPU usage proportions?

       

      Thanks,

      Hitesh

        • 1. Re: CPU usage by project/department  - chargeback criteria
          Brian Coggon

          Hi Hitesh,

           

          I can not answer all of your question but I can address a couple of things that your post entails.

           

          I am not sure how other IT departments are handling and accounting for these types of actions.  Your question opens the idea that by allocating processor and storage time to a project you could assess a fee to the departments and also use this method to help plan for the cost of expansion.

           

          There are many different ways to try and assess this cost I would not recommend monitoring the CPU as this will add cost and latency to the tableau server process. You may just try and allocate based on number of projects or size of projects.

           

          Regarding the allocation of a CPU for each action this is not quite correct.   We try to optimize the CPU so I don't think you can assume that one action will have a linear process on one CPU.

           

          Hope this is helpful.

           

          Cheers,

          Brian

          • 2. Re: CPU usage by project/department  - chargeback criteria
            lei.chen.0

            Hello Hitesh,

             

            For charge back criteria, I suggest organizing by site and charging by user count.

             

            By organizing by site: you can set storage limit and user limit, however CPU / memory limit setting is not possible.

            By user count: this is reasonable for Tableau license is charged by user. You may refer to Tableau online user charge fee pattern and adjust it to your company situation.

             

            As for your out of CPU capacity with two users, it sounds strange.

            I suggest you investigate and find the reason first.

            For example, which process occupied the CPU most. Tableau Server Processes

             

             

            Regards.

            • 3. Re: CPU usage by project/department  - chargeback criteria
              Russell Christopher

              Hitesh -

               

              Tableau Server does not do govern and/or track resource usage at the level of granularity you are interested in. There is a vendor named Palette software who has figured out a way to do this, but this will cost extra money for you, of course.

               

              Brian mentions not monitoring the CPU because doing so adds cost and latency to the server. Maybe I don't understand what he's trying to say, but in the main, I don't agree. Tools like TabMon and PerfMon are perfectly fine and won't hurt anything as long as they are used in a reasonable and practical fashion If you're reading/storing CPU utilization every second, then that's probably overkill...

               

              You asked:

               

              - Is it just that we identify the time windows when CPU is running short (after other attempts of optimizing usage) and see who all are active in that window and approach with that data?

               

              It'd probably be easier just to watch for long running reports and extract refreshes that take a long time to compete - these will generally be the things taking the most CPU. Both of these situations are tracked in the standard Admin vizzes of Tableau.

               

              - Can it be assumed that each action occupies one CPU core for it's duration (Vizql, data engine, data server working together). Certainly not accurate, but can it give a good estimate of CPU usage proportions?

               

              No. Certain activities will be limited to a single core, but other activities will use all available cores (based on server settings) to complete as quickly as possible. Especially if you're going to be doing "costing" with this data, you need to be correct, or people will be very unhappy with you

               

               

              • 4. Re: CPU usage by project/department  - chargeback criteria
                John Abdo

                Hi Hitesh,

                 

                Russel is correct, as always.

                 

                Yes. CPU is the most critical limiting resource for serving Tableau visualizations, and as such it is the only reliable metric to do objective chargeback that your users will value. Other attempts to chargeback using workbook count, storage level, number of users, etc. all fail to accurately represent the real resource demands on your server, and as such smart business users can easily poke holes in that model push back on your attempts to justify your valuation of their cost on the shared Tableau Server.

                 

                You can learn more about how we solved this challenge by attending our webinar about Tableau Shared Services in the Enterprise on Tuesday, October 11th at 11am (San Francisco time), whereby I will demonstrate our Palette Chargeback reporting as well as our Palette Director, which can allow you to provision CPU cores by user group or usage type, such Web Authors in Marketing, Extractors in HR, or regular Interactors in Sales, etc.

                 

                Here is the link to register to our webinar.


                http://palette-software.com/events/webinar-shared-services

                Otherwise, feel free to contact us at hello@palette-software.com. Below is a real CPU Chargeback report by Project but the names have been anonymized with the Tableau repository IDs. In this example, the customer determined that each CPU core costs them $10 per hour to provide, so the dollar values are calculated by multiplying the total CPU Core Hours by $10/hour.

                 

                Cheers,

                John

                http://palette-software.com

                Palette Chargeback by Project CPU.png

                • 5. Re: CPU usage by project/department  - chargeback criteria
                  Toby Erkson

                  How would you account for payback if Guest access is used?

                   

                  Tableau is part of the databases admin group and thus gets a budget which covers the Tableau Server where I work.  It's used corporate-wide as it provides the service of business intelligence to the entire company.  The Desktop licenses are purchased by the groups that need it and they cover the yearly maintenance*.

                   

                  < soapbox >

                  This is a step forward because I think the prior system of charging back is out-dated.  It may have made sense back when hardware was expensive.  Data is what steers a business and penalizing people for wanting access to it and using it is counter-productive and counter-intuitive.

                  You really want to know who is taking this big step forward into the future and doing it right?  Ford Global-Data-Insight -Analytics

                  < /soapbox >

                   

                   

                   

                  * I would take it a small step forward and hope for floating licenses (licensing pool) so we can give them to those who only need to use Tableau for a short time as purchasing a full license couldn't be cost justified.

                  • 6. Re: CPU usage by project/department  - chargeback criteria
                    John Abdo

                    Hi Hitesh,

                     

                    I have just posted the video of the webinar of October 11, 2016, that I mentioned in my prior reply to your question. In addition, we also now have published a live demo dashboard of our Palette Chargeback solution so you can see exactly what kind of detailed reporting you can expect. This kind of insight is really essential for anyone who in managing Tableau Server in the Enterprise, especially if you have a core license.

                     

                    The dashboard has real, but anonymized usage data! It is very exciting to see just how a real Fortune 500 Tableau customer use the cores for Interactor visualization. Never before has this kind of data been made available for us to consider.

                     

                    If you have a moment, I recommend you check out my related reply post on the Forums regarding Workbook CPU measurement.

                     

                    Cheers,

                    John Abdo, CEO

                    john@palette-software.com

                    www.palette-software.com

                     

                    PS: While we are talking CPU, you will really want to know about our Palette Capacity solution which really makes super easy to understand your CPU and Memory capacity utilization. You can easily pivot from the cluster to the worker or even the process. Instantly remove all Non-Tableau CPU to verify if the spikes are Tableau related. Even more incredible, you can see the core utilization in super high resolution time series over the past 10 days at a glance. If you something that inspires you to understand more, then drill into anything using Tableau itself. It's really cool!

                     

                    Try the live version of this dashboard.

                    Screenshot 2016-10-27 20.11.16.png

                    1 of 1 people found this helpful