8 Replies Latest reply on Jul 28, 2016 6:34 AM by David Pires

    Maximum Limit Of Data upload in Tableau from database

    pardeep kumar

      Hi All ,

       

      i have installed Tableau server on  Window Server 2012 AND configuration are

      64 GB Ram

      8 Core Processor and 320GHz clock speed

       

       

      i want to know what is the maximum amount of data i can load into system Ram. I have 100M records in one table and if i select that table, does Tableau load all data or its going to hang out......?

       

      What is the best way to implement granularity like (5 minute,1 hour  and 1 day )

       

      One way to create different table for different granularity or it can be handle in one table....?

        • 1. Re: Maximum Limit Of Data upload in Tableau from database
          Steve Martin

          Hi Pardeep,

           

          This looks to be two questions. Please can you raise each on its own post.

           

          On your first question, Tableau can easily handle 100M records, I have loaded in over a billion on a dev server with a similar setup to you. In practice, this is something best handled with a direct connection or using something like a parameterised custom sql statement to reduce the amount of data for Tableau to handle or pre-aggregating.

           

          On your second question the best way to implement granularity is to use:

           

          ...Exactly, we do not know your use case or data so are unable to provide you with a recommendation.

           

          Steve

          • 2. Re: Maximum Limit Of Data upload in Tableau from database
            David Pires

            Steve is right, 100m shouldn't be a problem.

             

            But that will depend on the database. Where are connecting it Tableau to? Live or extract?

             

            There are a number of factors in play that may hinder performance.

             

            David

            • 3. Re: Maximum Limit Of Data upload in Tableau from database
              pardeep kumar

              Hi David,

               

              Thanks for reply

              i'm using Live connection because i need to provide custom auto refresh where user can select auto refresh interval time and after that dashboard refreshed using  Tableau JavaScript api ,so extract method not work here because we can update extract only using scheduler not on user run time request update.

               

              and one more help i need , i have three tables 5minutegranularity, 1hourgranularity and 1daygranularity so can i use these three table on one report like if user select 5 minute option it fetch data from 5 minute table or if he changed to 1 hour then it fetch data from 1hour table...?

              • 4. Re: Maximum Limit Of Data upload in Tableau from database
                pardeep kumar

                Please look in attached excel file you can understand what i want..?

                • 5. Re: Maximum Limit Of Data upload in Tableau from database
                  Steve Martin

                  Your attached Excel file doesn't make your granularity requirement any clearer other than your volumes are aggregating exactly as expected.

                   

                  I have to ask, do you know which grain you have been asked to provide? This would provide you with a starting point. That said, if possible, it would be best to drop to the lowest grain possible and review at a later date if needed.

                   

                  Steve

                  • 6. Re: Maximum Limit Of Data upload in Tableau from database
                    David Pires

                    Hi Pardeep,

                     

                    A live connection is fine and it shouldn't be a problem. Therefore I'm not clear why would you need an auto-refresh. If your database gets updated your data gets updated at the same time.

                     

                    With regards with various levels of granularity you can always use 3 distinct data sources/connections to separate these.

                     

                    David

                    • 7. Re: Maximum Limit Of Data upload in Tableau from database
                      pardeep kumar

                      Hi David Pires,

                       

                      I need auto refresh because user requirement is when he open a report in browser using chrome or Mozilla or IE and he not doing any activity on report or browser its in idle state  but data in report needs to be updated after user selected time interval if any new record insert in table.

                       

                       

                      and second can you explain it more or any reff link..?

                      With regards with various levels of granularity you can always use 3 distinct data sources/connections to separate these.

                       

                      Thanks or you help and quick reply.

                      • 8. Re: Maximum Limit Of Data upload in Tableau from database
                        David Pires

                        What I meant was that you could create a query to your data source showing only the 5 minute table. Then do the same for the other refresh intervals.

                         

                        If you can find a way to refresh the datasource itself you can specify various intervals.

                        Otherwise refresh all at the lowest interval but the other two will only change when your database has been updated. This may not be ideal depending on the size of the database and complexity of your report.

                         

                        David

                        1 of 1 people found this helpful