8 Replies Latest reply on Jun 29, 2015 6:59 PM by Donald Leung

    Redshift and Tableau Online storage

    Donald Leung

      Hi, I am now studying about Tableau and I have questions on the product.

       

       

      I am now design the architecture of the whole system, each year we will collect 3 TB of data.

       

       

      We plan to put the 3TB/year data to Redshift in Amazon.

       

       

      And I prefer Tableau online to Tableau server, as it saves me cost on buying and maintaining the hardware.

       

       

      My question is, the Tableau online has a limit of 100GB storage, what does that actually mean?

      As the data actually comes from Redshift, so theotically I am probably not using any of the 100GB storage at all? Is my understanding correct?

       

       

      Also, how do you compare Redshift and Big Query from Google? Which one works better with Tableau at the moment?

       

       

      And,  where datacenter should choose to place my Redshift server? Which one is closest to the Tableau online servers?

       

       

      Lastly, what should I expect for the query performance in such system with 3TB of data? What is the mean wait time when i operate using the Tableau desktop?

       

       

      Sorry for having so many questions at a time.

       

       

      Thank you very much for your help.

        • 1. Re: Redshift and Tableau Online storage
          Dustin Smith

          Hi Donald,

           

          Great questions!  Let me try and answer them in order:

           

          1.

          the Tableau online has a limit of 100GB storage, what does that actually mean?

          As the data actually comes from Redshift, so theotically I am probably not using any of the 100GB storage at all? Is my understanding correct?

           

          Answer:  You are correct.  If you are storing your data in Amazon Redshift then you will be able to establish a live connection between Tableau Online and your Redshift deployment.  This means that no data is taken from Redshift and stored in your Tableau Online account.

           

          2.

          Also, how do you compare Redshift and Big Query from Google? Which one works better with Tableau at the moment?


          Answer:  Both Amazon Redshift and Google BigQuery work great with Tableau Online.  Customers seem to choose one over the other for specific reasons relating to the type of project they are working on.  Many time it also comes down to factors like how familiar they are inside an Amazon Web Services environment vs. a Google Cloud Platform environment, etc.


          3.

           

          And,  where datacenter should choose to place my Redshift server? Which one is closest to the Tableau online servers?

           


          Answer:  This is a question you should probably discuss with your Amazon Web Services account manager.  I know it is common for customers using Amazon Redshift and Tableau Online to choose an availability zone for their Redshift deployment that is closest to where they or their customers are located.


          4.

           

          Lastly, what should I expect for the query performance in such system with 3TB of data? What is the mean wait time when i operate using the Tableau desktop?

           


          Answer:  There is no single metric here given that there are a ton of factors in play.  Everything from how things are indexed in your Redshift deployment to what type of analysis you are trying to do in Tableau.  Overall though I think most people are shocked at the query speeds in Redshift, especially if they're used to trying to work with the same size of data in relational/transactional systems (like MySQL, etc.).  We have a whitepaper that we've published with Amazon specifically on performance tips for working with Redshift and Tableau.

           

          I hope these answers are helpful and please let us know if we can provide additional information.

           

          Best,

          Dustin



          • 2. Re: Redshift and Tableau Online storage
            Donald Leung

            Hi Dustin,

            Thank you very much for your great answers.

             

            Sorry that I have 2 more questions:

            (1) I am now looking at the reliability of the Tableau online service and I find this page:

            https://trust.tableau.com/status/tableau-online

            It shows that both 10az.online.tableau.com and 10ay.online.tableau.com has problem on 21June, I wonder what happened to them?

             

             

            (2) I am also concerned about the after sales support, if I go for tableau online, I wonder what support service will I get?I read the page: Support Services | Tableau Software and I am not sure what support will I get for Tableau online.

             

            Thank you very much.

             

            Regards,

            Donald

            • 3. Re: Redshift and Tableau Online storage
              Dustin Smith

              Hi Donald,

               

              Awesome that you're looking at Tableau Online and really doing due diligence.

               

              I'm really glad that you found trust.tableau.com.  Transparency is a big deal to us and we want everyone to know our track record for the Tableau Online platform.  If you click on the little red icon for 21 June there is a pop-up that will display and given you a run-down of event for that day.  Looks like we had a very short 9 min. service interruption.

               

              In terms of support for Tableau Online after you become a customer, that is actually something we are really proud of.  You'll have the same level of support from multiple teams just like you would if you purchased any other product from Tableau (your account manager, your customer success representative, AND our technical support team).

               

              If you're interested in having a deeper discussion around how we see folks ramping up quickly with Tableau Online and engaging with our Support Team, I'm happy to help coordinate a meeting.

               

              Feel free to email me directly any time: dsmith@tableau.com

               

              Best,

              Dustin

              1 of 1 people found this helpful
              • 4. Re: Redshift and Tableau Online storage
                Donald Leung

                Hi Dustin,

                Thank you very much for the reply.

                 

                I have one more question, it is that for the 100GB on Tableau online, is it compressed data or raw data?

                 

                Thank you very much.

                 

                 

                Regards,

                Donald

                • 5. Re: Redshift and Tableau Online storage
                  Dustin Smith

                  Hi Donald,

                   

                  The only way to physically store data on Tableau Online is actually to compress it by taking a Tableau Data Extract (.tde files).  The compression for .tde's is actually pretty mind blowing.  As a guy who used to have to manage some pretty large .csv and Excel files, I about fell out of my chair the first time I extracted a 100MB Excel file and it shrank to under 30MB.  That was when our extracts first came out more than 4 years ago and since then the compression technology behind it has gotten even better.

                   

                  Long story short, I don't think you need to worry about the 100GB storage limit for Tableau Online.

                   

                  Follow-up question on my end:  Are you looking at extracting from Redshift or from other data sources to use along side Redshift?

                   

                  Best,

                  Dustin

                  1 of 1 people found this helpful
                  • 6. Re: Redshift and Tableau Online storage
                    Donald Leung

                    No, I am probably not extracting the tde file from the database. However, I still would like to know the capability of the system in case i need it, or my colleagues ask me that.

                     

                    Sorry that I dont quite get the answer, so does that mean it is 100GB of compressed data? That means I could store around 300GB of raw data to Tableau Online if the compressed data happens to have a compression ratio of 3?

                     

                    I have to know this as I might really get a lot of data to process (3TB/yr).

                    • 7. Re: Redshift and Tableau Online storage
                      Dustin Smith

                      Hi Donald,

                       

                      I think for the purposes of your exercise the answer is yes, 100GB of storage on Tableau Online is compressed data stored in a Tableau Data Extract file.

                       

                      The best way to understand what your compression ratio will be is to do a 30 second test using a small representative sample of the data you'll be using (last 10 days worth or something).  If you can get it into a .csv and note the files' size, then open it in Tableau Desktop and extract it (no aggregation, full granularity) and see how big the .tde file is.  The difference in size should give you a ballpark of the compression you'll see.

                       

                      *note:  .tde files have a min. size just based on the internal components that allow for compression and fast read, so if youe extracted a 10 KB .csv, the .tde might actually be bigger to account for the basic footprint every .tde has.  I'd start with a file greater than 10MB if possible.

                       

                      Best,

                      Dustin

                      1 of 1 people found this helpful
                      • 8. Re: Redshift and Tableau Online storage
                        Donald Leung

                        Hi Dustin,

                        Thank you very much for your reply.

                         

                         

                        The aggregation you mentioned arouses my interest.

                         

                         

                        So for the Tableau extract file, it could do aggregation for me.

                         

                         

                        I wonder if such aggregation is available on Redshift database, or other DB choices?

                         

                         

                        As I have been looking for ways to aggregate data, it would be great if Tableau could also help on this.

                         

                         

                        Thank you very much Dustin.

                         

                         

                         

                         

                         

                         

                        Regards,

                        Donald