1 2 3 4 Previous Next 84 Replies Latest reply on Mar 30, 2015 6:23 AM by Cristian Vasile Go to original post
      • 15. Re: Would a RAM disk improve Server performance?
        Matt Coles

        Yes, you're right, I should do my own homework on those questions--sorry. I figured you would already have those answers though, as you are also quite knowledgeable about Server, and it seemed like you had thought through the ideas quite a bit. Also--while it's true that I probably have better access to answers from our developers than most other customers (we consider ourselves a customer as well), it's not always as direct as you might think (especially as we near the release of a new major version!).


        But, I vow to provide answers to these questions. And if it's thought by Dev that some of these techniques may still yield substantial performance improvements in a post-9.0 world, I will see about trying the RAM disk idea out on some test servers, and posting my results.

        • 16. Re: Would a RAM disk improve Server performance?
          Cristian Vasile

          Hello Matthew,

           

          Thanks a lot for the reply, unfortunately I don't have free resources (time, hardware, etc) now to run some tests especially with superspeed's supercache application which looks in my opinion promising.

           

          Regards,

          Cristian.

          • 17. Re: Would a RAM disk improve Server performance?
            Toby Erkson

            I played around with ImDisk Virtual Disk Driver.  Came to a "duh" moment:  The pointing of TMP and TEMP to a RAM disk won't get touched so long as there is physical RAM for the server.  The only way I was able to get a temp file to show up on my RAM disk was when I pretty much took all available RAM and made it into the RAM disk.  Then it started getting used but the system didn't seem to like being so super low on RAM and things slowed down anyway, so it was a no-win situation.  Am I right?

             

            This got me thinking about what y'all and our account managers have told us and for improved performance you need to have faster disk access.  So it makes sense to me that to get the performance I/O increase I need to put the Tableau Server install on a RAM disk, right?

             

             

             

            Self-note

            Propagating Windows environ variables:  http://serverfault.com/questions/8855/how-do-you-add-a-windows-environment-variable-without-rebooting, this answer:

             

            environmental_variable_refresh refresh_environmental_variable refresh environment path path_variable

            • 18. Re: Would a RAM disk improve Server performance?
              Jeff Strauss

              Is it true that temp and tmp paths don't get touched unless the app runs out of memory?  I've been wondering about this because I have windows installed on C, and Tableau installed on G, and the temp files pointing at C.  I was told at one point (a long time ago) from Tableau prof services to repoint the temp paths to the G drive, but I have not.

               

              Also, I saw the attached article, but it didn't make sense to me as to when the temp path is utilized for Tableau:

              Changing the Default TEMP Location for Creating Extracts | Tableau Software

               

               

              Thanks for any pointers on here that you may have.  In terms of the question you have on data extract location, there is a non documented redirect of the Tableau data extract path so instead of pointing it at the default location, you can customize, though it may not be supported.  Not sure if it still works or not, but you can try it out on your sandbox.  tabadmin set dataengine.database_dir “dataengine location

              • 19. Re: Would a RAM disk improve Server performance?
                Cristian Vasile

                Toby,

                 

                Please, tell me that you are playing with a spare server!

                 

                Do not allocate all RAM to ram disk, because OS will start swapping as mad.

                 

                Now, please check the temp folder used by tableau stack

                Tableau Server uses the folder :\ProgramData\Tableau\Tableau Server\data\tabsvc\temp

                So please let us know what values of TMP and TEMP variables did you modify?

                 

                " I need to put the Tableau Server install on a RAM disk, right?"

                 

                NO, leave the software where it is now,  you should try to modify TABLEAU server temp directory to point to R:\  where R: is the letter assigned to your ram disk.

                 

                If you manage to trick tableau to use ram disk space for temporary files & extracts then try to refresh some .TDEs, the largest ones for example.

                 

                You have too many extracts to place them on ram disk, and is too dangerous to do this. You can update the extract on hdd and forget to copy it on production / working space, and internal customers will run vizzes against old data.

                 

                Regards,

                Cristian.

                • 20. Re: Would a RAM disk improve Server performance?
                  Cristian Vasile

                  Jeffrey,

                   

                  The article you have publish is for Tableau Desktop.

                  http://www.tableausoftware.com/tableau-data-extracts-part3

                   

                  Quote

                  Tableau Desktop uses the location pointed to by the system’s %TEMP% environment variable for all temporary files used in the process of creating extracts.

                  Tableau Server uses the folder :\ProgramData\Tableau\Tableau Server\data\tabsvc\temp

                   

                  Hope this helps.

                   

                  Regards,

                  Cristian.

                  • 21. Re: Would a RAM disk improve Server performance?
                    Cristian Vasile

                    Sounds logical for me.

                     

                    Tableau Data Extracts - Tips, Tricks and Best Practices | Tableau Software

                    • In a Tableau Server environment, it’s important to make sure that the backgrounder has enough disk space to store existing Tableau extracts as well as refresh them and create new ones. A good rule of thumb is the size of the disk available to the backgrounder should be two to three times the size of the extracts that are expected to be stored on it.
                    • Tabcmd (a command-line utility) can be used to refresh extracts, as well as to publish TDEs to Tableau Server

                     

                    Regards,

                    Cristian.

                    • 22. Re: Would a RAM disk improve Server performance?
                      Jeff Strauss

                      Cristian, thanks for the clarification!

                      • 23. Re: Would a RAM disk improve Server performance?
                        Cristian Vasile

                        Toby,

                         

                        The reason i suggested you to try a small ram disk is simple, i suppose that vizqlserver.exe  and/or data engine (tdeserver64.exe) will need temporary disk space to be able to push correct data back to clients.

                        The list with all tableau processes http://onlinehelp.tableausoftware.com/current/server/en-us/help.htm#processes.htm%3FTocPath%3DAdministrator%2520Guide|Troubleshooting|Work%2520with%2520Log%2520Files|_____1

                         

                        Regards,

                        Cristian.

                         

                        data engine

                        tdeserver64.exe

                        tdeserver.exe

                        Stores data extracts and answers queries

                        Yes

                        The data engine's workload is generated by requests from the VizQL Server process. It is the component that loads extracts into memory and performs queries against them. Memory consumption is primarily based on the size of the data extracts being loaded. The 64-bit binary is used as the default on 64-bit operating systems, even if 32-bit Tableau Server is installed. The data engine is multi-threaded to handle multiple requests at a time. Under high load it can consume CPU, I/O, and network resources, all of which can be a performance bottleneck under load. At high load, a single instance of the data engine can consume all CPU resources to process requests.

                         

                        VizQL Server

                        vizqlserver.exe

                        Loads and renders views, computes and executes queries

                        Yes

                        Consumes noticeable resources during view loading and interactive use from a web browser. Can be CPU bound, I/O bound, or network bound. Process load can only be created by browser-based interaction. Can run out of process memory.

                        • 24. Re: Would a RAM disk improve Server performance?
                          Toby Erkson

                          Cristian Vasile wrote:

                           

                          Toby,

                           

                          Please, tell me that you are playing with a spare server!

                          Yes, of course.

                           

                          Do not allocate all RAM to ram disk, because OS will start swapping as mad.

                           

                          Now, please check the temp folder used by tableau stack

                          Tableau Server uses the folder :\ProgramData\Tableau\Tableau Server\data\tabsvc\temp

                          So please let us know what values of TMP and TEMP variables did you modify?

                           

                          I'll need to redo this.  I had changed the below variables in the red rectangle (those aren't the actual values I used for testing):

                           

                          " I need to put the Tableau Server install on a RAM disk, right?"

                           

                          NO, leave the software where it is now,  you should try to modify TABLEAU server temp directory to point to R:\  where R: is the letter assigned to your ram disk.

                           

                          If you manage to trick tableau to use ram disk space for temporary files & extracts then try to refresh some .TDEs, the largest ones for example.

                           

                          You have too many extracts to place them on ram disk, and is too dangerous to do this. You can update the extract on hdd and forget to copy it on production / working space, and internal customers will run vizzes against old data.

                           

                          Regards,

                          Cristian.

                          So why is my logic flawed regarding putting Server on a RAM disk?  I was going to uninstall Tableau Server from the HDD, install it on the RAM disk, then restore from a backup.

                          • 25. Re: Would a RAM disk improve Server performance?
                            Cristian Vasile

                            It is flawed, you need to accelerate intrinsic processes, the "data pipeline" and not the code (*.exe);  please read my last post to understand the workflows and where the bottlenecks could appear Keep in mind that tableau stack use ~20 folders to store logs!

                            Server Log File Locations

                            It is not wise to move all stack on ram disk  to fill your precious ram disk with unnecessary log files.

                             

                            Regards,

                            Cristian.

                            • 26. Re: Would a RAM disk improve Server performance?
                              Cristian Vasile

                              OK, being on a spare machine, we can play games for all T-admins, the final test will be to move all .TDEs and associated files on ram disk and start generate vizzes.

                               

                              Regards,

                              Cristian.

                              • 27. Re: Would a RAM disk improve Server performance?
                                Cristian Vasile

                                Toby & Jeffrey,

                                 

                                Well, I found the math equation who tell us how much free space on temp folder we need!

                                 

                                Refreshing extracts

                                If using extracts, consider the space needed by the Temp directory during an extract refresh. The Temp directory, which is where an extract is stored to during a refresh, may require up to the square of the final file size of the extract. For example, a 12 GB extract may take up 144 GB of disk space to complete the refresh.

                                 

                                Understanding Disk Space Requirements for Tableau Server | Tableau Software

                                 

                                Regards,

                                Cristian.

                                1 of 1 people found this helpful
                                • 28. Re: Would a RAM disk improve Server performance?
                                  Cristian Vasile

                                  Toby,

                                   

                                  I have a new idea, should work if we use symbolic links to trick tableau.

                                  NTFS symbolic link - Wikipedia, the free encyclopedia

                                   

                                  Regards,

                                  Cristian.

                                  • 29. Re: Would a RAM disk improve Server performance?
                                    Toby Erkson

                                    Cristian Vasile wrote:

                                     

                                    OK, being on a spare machine, we can play games for all T-admins, the final test will be to move all .TDEs and associated files on ram disk and start generate vizzes.

                                     

                                    I set up a 6GB RAM disk (F: drive for Fast since my R: is already being used), changed the system variables TEMP and TMP to point to the F: drive, propigated the Windows variables, and restarted the Tableau Server.  Ran an extract and the F: drive isn't being touched, everything is going to the default temp folder you stated, D:\Application\Tableau\Tableau_Server\data\tabsvc\temp   I've got a 90 minute extract running right now and I can see the folder being populated, updated, folders created and deleted, etc.  Workbook size is 1.2MB.

                                     

                                    FYI:  Corporate policy is to have our applications put on a separate drive from the OS, thus why the file path begins with D:\Application.

                                    1 2 3 4 Previous Next