3 Replies Latest reply on Jun 23, 2015 4:06 AM by Martin Luxhøj

    TDE file storage when using the data server

    Tom Brown



      I am seeing fast growth of the space required by my Tableau server, and have looked into the folder structure to see where this is happening.


      In the dataengine folder I see folders with names like folder_1234 which the contain other folders like upload_{guid}.


      Each guid folder then seems to contain a TDE file - which makes sense.


      The problem I have is that I now have 900GB of these files, and only have about 14 extracts on the server (they are big though, 500M rows of itunes data for example).  But there is repetition of the TDE files in the folder structure.


      Should I be running some maintenance tasks to get rid of these replica folders?  perhaps backup does it, which has not yet been run on this server.


      Any ideas?


      Thanks in advance.



        • 1. Re: TDE file storage when using the data server
          Tracy Rodgers

          Hi Tom,


          There is a Reap Extracts background task that should be removing repetitive TDE files. The reap process occurs on a regular schedule. Until the reap occurs, old extracts will still be in the extract engine and will take up additional space. An extract will not be cleaned up unless it is really no longer needed – i.e. no workbooks will reference it anymore, and no active viewer sessions reference it as well. A reap of data extracts on Tableau Server occurs under 3 scenarios:

          1. A workbook is deleted. The reap will happen if there are not any active vizql sessions referencing any workbooks that use the extract. Then there is a hold off period of several hours after the last session to use an extract is gone before reaping will occur.

          2. An extract refresh has occurred. Old extracts that have been replaced by a newer refreshed extract will get reaped after the vizql session created by the refresh background job expires, and then after a hold off period of several hours.

          3. During the backup process. While the backups are taking place, a “reap” process is performed, which cleans up an extract that is no longer in use.


          I believe that there was a known issue that this wasn't working in some of the earlier 7.0 versions--this being said, I would recommend upgrading to 7.0.4 that was released last week, if you haven't already.


          Hope this provides some insight!



          2 of 2 people found this helpful
          • 2. Re: TDE file storage when using the data server
            Tom Brown

            Thanks Tracy.


            Are you able to confirm the V7 versions affected by this?  I guess this might be the problem I'm seeing as there is way too much space being consumed by the extracts on the server I'm using.





            • 3. Re: TDE file storage when using the data server
              Martin Luxhøj



              I know this is a VERY old thread but I'm experiencing the same problems as Tom describes....but I'm running version 8.3.


              I think my problems started after I installed version 9. I had too many issues regarding performance and disk usage which I didn't have running version 8.0. So I uninstalled version 9 and installed version 8.3 and I have had problems ever since.


              Problems include:


              • Disk usage
              • Subscribtions that won't "trigger"


              The Tableau Server (32bit) is running on a virtual machine (WMware) 64 bit windows 2008 R2 Standard


              Any ideas?