7 Replies Latest reply on Sep 26, 2016 9:17 AM by Federico Armentano

    PC FOR LARGE DATA SETS

    Federico Armentano

      Hi, I would like to know your experience with large data sets. I am thinking that working with really big data sources (for example, 30 GB) with my laptop will not work correctly.

      That why I would like to know your experiences with large data sources and what characteristics must be to be considered a the time of buying a computer.

      Thanks all!

        • 1. Re: PC FOR LARGE DATA SETS
          Carl Slifer

          Hi Federico,

           

          A few questions to help get people started. Are you looking to store that data on the laptop or read from a server or another repository somewhere?

          What will you be doing with the data? Is there a reason to be using it all at once?  Is your data numerical? Text? Etc?

           

          There's just so much to think about and so little to go on from here. Someone more technically adept might be able to give you specifics but I can at least help you prep answers to the questions will be coming in.

           

          Best Regards,

          Carl Slifer

          InterWorks

          1 of 1 people found this helpful
          • 2. Re: PC FOR LARGE DATA SETS
            ra.sae.0

            Hi Federico,

             

            I dont have the exact size of my database. Few were about 300 Million records with 15-20 Dimensions & 10 Measures,

            70 Million records with similar number of dimensions & 10 Measures.

             

            Execution time Takes about 20-30 seconds for the first source and 10-15 seconds for the latter. Considering usage of Complex calculations in them.

             

            The RAM was 8 GB. It took some time though. Having higher RAM will be helpful.

            And sometimes you could tune the performance of your dashboards by Hiding the unwanted dimensions, filtering out the data. You find many Performance tuning tips around.

             

            My suggestion is go for PC with 8+ RAM.

             

            Hope this helps

            3 of 3 people found this helpful
            • 3. Re: PC FOR LARGE DATA SETS
              kettan

              I have no experience with this and therefore can't really answer.

               

              I am just curious if you are going to store your data in a database and if so what kind.

              If yes, it is perhaps equally important to check out what the database needs to perform well.

              • 4. Re: PC FOR LARGE DATA SETS
                kettan

                Thanks. This is helpful.

                 

                Just curious, what database are you using?

                 

                I am asking, because performance will also depend on what kind it is.

                • 5. Re: PC FOR LARGE DATA SETS
                  ra.sae.0

                  Thanks Kettan SQL Server, but we use Extract.

                   

                  The data are so granular and huge, hence the performance is very bad for few large datasets.

                  1 of 1 people found this helpful
                  • 6. Re: PC FOR LARGE DATA SETS
                    kettan

                    Thank you 

                     

                    I will follow this interesting thread with the hope that others also share their experience.

                    1 of 1 people found this helpful
                    • 7. Re: PC FOR LARGE DATA SETS
                      Federico Armentano

                      Hi all, thanks for participate in this discussion. I have been researching and I would like to share it with the community.

                      My data base is 65 columns (10 strings and 55 measures) and more than 20 Million rows (and adding records month by month).

                      My actual computer is a Lenovo T440.

                      Processor: Intel (R) Core (TM) i5 – 4300U CPU @ 1.90 GHz 2.49  GHz

                      Installed memory RAM: 8 gb (7,69 usable) - 64-Bit

                      According my research and some IT friends advices, I should consider:

                      • adding memory RAM: 16 gb,
                      • Processor i7,
                      • The most important. Even Tableau users should consider it before looking for a new computer. Adding a SSD (Solid-State drive) will improve our performance.
                      • We can ignore special video qualities, which are not necessary.

                      Hope this helps,

                      3 of 3 people found this helpful