3 Replies Latest reply on Mar 8, 2017 1:05 PM by Michael Damico

    Improving performance loading large data sets in text tables (Tableau Server)

    Tom Cody

      Hi everyone, I think I may being getting into "this isn't what Tableau is for" territory, but my user base loves it, so any insight is appreciated. I'm loading a lot of data into a text table on Tableau Server (12 columns, ~250k rows.... Server has 8 cores and 48GB RAM). When clicking into this view, it generally takes ~5 minutes to load and any subsequent filtering drags as well. If the server is under stress, sometimes it doesn't load and we get a server error and reset view alert. The data is all coming from an extract. The reason I need to display so much unfiltered data at once is because there are hundreds of users, all looking for something a little different (and I can't predict exactly what they are looking for so I can't create multiple workbooks). Ideally, I'd like to continue loading all of this data at once. I experimented with adding an index and only displaying ~10k rows at one time with a filter, but users complained.

      A few questions:

      • Is it even possible to improve the performance of a task like this, or is it just something Tableau shouldn't be doing? If it is possible-
      • Is there a scenario where using a live connection instead of an extract would make this faster? Right now I'm using SQL, but is there a particular platform that is just very fast getting queried and tossing data to Tableau?
      • Is it just a matter of horse power? Do I just need to add more and more RAM? Or maybe just increase disk I/O?
        • 1. Re: Improving performance loading large data sets in text tables (Tableau Server)
          Michael Damico

          So that I understand, are you saying that you're dashboard is effectively loading a viz that has 250k rows of data for someone to scroll through? As for your questions:

           

          • Is it even possible to improve the performance of a task like this, or is it just something Tableau shouldn't be doing? If it is possible-
            Can you aggregate the data up to a dimension level to start and let users drill down into it? Ideally Tableau wants to show viz's, not text tables. We struggled with this and eventually told users the only way to improve performance is to change to viz's and start with less data
          • Is there a scenario where using a live connection instead of an extract would make this faster? Right now I'm using SQL, but is there a particular platform that is just very fast getting queried and tossing data to Tableau?
            Not that I'm aware of. Extracts have always been faster in my experience. There are always things like network latency, query time, and whatever overhead Tableau has to deal with to take into consideration
          • Is it just a matter of horse power? Do I just need to add more and more RAM? Or maybe just increase disk I/O?
            Unlikely, those specs should be able to satisfy your number of users, I think it's in the way the data might be being displayed
          • 2. Re: Improving performance loading large data sets in text tables (Tableau Server)
            Tom Cody

            Correct, just a viz that shows 250k rows of data.  250k is an extreme, generally it is 5-10k.  I realize this isn't what Tableau was built for, but the users love it (besides waiting for the data to load).  This is a viz for a sales team.  They use one tab to identify penetration gaps, and another tab to get details on that gap (products sold, who sold it, when it was sold, phone numbers, geographies, etc) so the details are extremely important... no way to really aggregate it to less detail. 

            • 3. Re: Improving performance loading large data sets in text tables (Tableau Server)
              Michael Damico

              5-10k isn't that much...and you're saying it's just a flat file of data, yes? There must be something else at play. I just took a 10k row flat file with 100 columns and sucked it into Tableau Desktop, deployed to Tableau Server and it's lightning fast. What do your filters look like? Do you have a workbook you can share that I can help diagnose?