1 2 Previous Next 16 Replies Latest reply on Oct 28, 2015 8:27 AM by Eric Allen

    tde file lock on Tableau server shared network drive

    Eric Allen

      Why is my tde file on a Tableau server shared network drive locked when all workbooks associated with the file are closed?  I am unable to overwrite using Alteryx or delete the file.

       

      Thanks,

       

      Eric

        • 1. Re: tde file lock on Tableau server shared network drive
          Toby Erkson

          You may want to move this to the Server Administration forum if you don't get any response here.

          • 2. Re: tde file lock on Tableau server shared network drive
            Justin D'Cruze

            We ran into a similar situation and haven't really found a solution other than to start publishing using the "Include External Files" option ticked.

             

            Check out the tdeserver64.exe processes on your Tableau Server machine with something like Process Explorer......you'll find that one of them will have a file handle open to your locked TDE file.

            • 3. Re: tde file lock on Tableau server shared network drive
              Arnold Slabbekoorn

              Does anyone have a solution for this. I am running in the same issue. I am trying to refresh the TDE file from Altered on the file share and it can't overwrite as the datasource connection on server keeps a lock on it. I don't want to directly publish to server as I am organizing dimension and metrics in folder structures and have calculated field build in the tds I uploaded through desktop. As my tde is 1 to 2 GB in size I do not want to constantly upload it manually.

              • 4. Re: tde file lock on Tableau server shared network drive
                Dan Scott

                I do not believe there is currently a solution to this.

                 

                I can say that extract files in the File Store are assumed to be immutable.  Refreshing an extract, for example, does not actually change the extract file---Instead it creates a brand new one, and the twb file stored in the Repository gets rewritten to refer to the new extract file, rather than the old one. Because of that, the lock doesn't normally matter.

                 

                If you actually altered an extract file in the File Store through ordinary file system operations, the File Store would not know that it needed to re-sync that extract to any other File Store nodes.  Maybe that's not what you are doing, but it kind of sounded like that.

                 

                We wanted to rename folders containing extract files, but couldn't do that either.

                • 5. Re: tde file lock on Tableau server shared network drive
                  Arnold Slabbekoorn

                  I think you are right Dan, no matter what I have tried, I can't get it to work. I ultimately just reverted back to a manual process, I don't have Alteryx server so I need to run workflows manually anyway right now. So what I did is run a workflow in Alteryx to generate a TDE file on the C: drive. Then open that TDE in an empty workbook in Tableau. Model everything I need like datatypes, folder organization, calculated metrics, dimensions and hierarchies. Save that as a tds data source in My Tableau Repository. Then uploaded the data source to the Tableau server.

                  Next I would again run the Alteryx workflow, which overwrites my TDE file locally. Then I would open again an empty Tableau workbook, but open on the left my saved tds file as saved data source and publish again the entire data extract to the server.

                  A bit cumbersome but in all the ALteryx workflow will run in the background and I get an email once finished. Then the process of opening the saved datasource and publish to the server only takes about 1 to 2 minutes weekly.

                  Advantages: It keeps all the meta data and modelling work I did in Tableau Desktop so when my other developers connect to the datasource in Tableau server to build their dashboards they retain the model and get weekly the data refreshed in their published server dashboards.

                   

                  I hope Tableau finds a solution for locked file share TDE files as extract data source refresh. An alternative I could build is just write the result in Alteryx back to our Datawarehouse as massive table but our DBA's are not so keen on that.

                  • 6. Re: tde file lock on Tableau server shared network drive
                    Dan Scott

                    I have requested that this behavior be changed, and I know the appropriate people are aware that it is not ideal.  For now, though, I think working around it is all we can do.  It sounds like you have figured out something that works for you.

                     

                    By the way, the locking is not an unmitigated negative. In some scenarios, views involving that extract will be served more quickly, as a result of the activity that is holding the lock.  Ideally, though we should not hold the lock immediately upon publish or refresh.

                    • 7. Re: tde file lock on Tableau server shared network drive
                      Zach Leber

                      Our solution to locked TDE files is to publish them to Server rather than have Server load them from the network. Then there is no problem when updating the datasource at the same time Server is using it.

                      1. install tabcmd

                      2. generate TDE (e.g. mydata.tde)

                      3. tabcmd login...

                      4. tabcmd publish mydata.tde -n mydata --overwrite --project...

                      5. connect your workbook in Desktop to the published datasource rather than the network datasource

                      • 8. Re: tde file lock on Tableau server shared network drive
                        Eric Allen

                        Do you lose the metadata (aliases, caluclations, etc.)  associated with the datasource on server when overwriting the tde using this method?

                        • 9. Re: tde file lock on Tableau server shared network drive
                          Zach Leber

                          I think you would lose the metadata unless it can be added to the TDE when it is generated using the Tableau SDK/API. We only have data in our TDEs.

                          • 10. Re: tde file lock on Tableau server shared network drive
                            Arnold Slabbekoorn

                            Not sure if you were referring to Alteryx but this is the only way I have been able to get the alias, etc to work:

                             

                            1. Alteryx output TDE file to a file share or local folder on your machine.

                            2. In an empty workbook I originally would point to the TDE file and build the meta data I want (folder organization, alias names, calculated fields etc)

                            3. Save the data source as TDS file on your local machine

                            4. Publish the data source to server from your local machine including all the meta data enhancements

                            5. When creating new workbooks point to the Tableau server data source

                             

                            I am running a weekly file share TDE refresh from Alteryx, I am not running it to server as it will overwrite my tds meta data model and revert back to the original column names and not having the calc fields or folder organization as ALteryx obviously can not manage that meta data portion.

                             

                            I then open an empty workbook and go to saved data sources (my local tds file) and simply republish my data source to the server. With alteryx email notification I can do other work while workflow runs and republishing takes 1 minute a week so all my workbooks pointing to the data source on server are refreshed.

                             

                            This is the only way I am able to run TDE files with a proper meta data model I can shared with other developers. Two other limited options:

                            1. Alias properly your columns in ALteryx, write all your calc fields in ALteryx, you don't care about folder structure for your developers and just want one list of measures and one list of dimensions. Then you can publish your TDE straight from Alteryx to Tableau server using the server upload macro's in the Power tools for Tableau package.

                            2. Build all Meta data in your local workbook/dashboard twb file and just copy that version if you want to make addittional ones based on the tde data source on the server. Dis advantage is that if you want to make changes to alias or calculated fields you have to fix each twb file individually as you lack a shared tds structure on server to utilize.

                             

                            Hope this helps, I hope ALteryx and its 3rd party developer in the future could build the folder meta data organization in Alteryx as well (Folder column and Column type (Dimension vs Measure) in the output tool so we can upload it directly from Alteryx.

                            • 11. Re: tde file lock on Tableau server shared network drive
                              Justin D'Cruze

                              You can avoid losing metadata such as calculations, hierarchies, etc, by using the --replace switch instead of --overwrite (this is what we do for our SDK based extracts)

                              • 12. Re: tde file lock on Tableau server shared network drive
                                Zach Leber

                                Justin, could you explain your workflow? When/how do you add the metadata to the disk-based TDE?

                                • 13. Re: tde file lock on Tableau server shared network drive
                                  Cristian Vasile

                                  FYI.

                                  Alteryx Bolsters Its Leading Data Blending Capabilities For Tableau With Ability To Deliver Analytics Directly To Tablea…

                                   

                                  "When faced with an overwhelming amount of data and requests for new visualizations or dashboards, analysts have been forced to constantly update data files using slow processes that could take weeks. Now analysts can scale their use of Tableau by automating the update of TDE files used to drive what could be hundreds or even thousands of visualizations. These benefits are available whether organizations deploy Tableau Server within their infrastructure or take advantage of Tableau online."

                                   

                                  Regards,

                                  Cristian.

                                  • 14. Re: tde file lock on Tableau server shared network drive
                                    Justin D'Cruze

                                    Sure - the key is to use TDS files to save the metadata information.

                                     

                                    We do all of our necessary modelling such calculations, create a TDS file using the Add to Saved Data Sources option (for making changes down the track), and then publish that Data Source up to Server via Tableau Desktop. This is a one-off step to get the metadata information up to Server.

                                     

                                    From that point onwards, every time the TDE file is updated (based on an external process) we run the TabCMD publish command via our automation tool

                                    E.g.

                                    "${tabcmd}" publish "${ExtractFile}" --name "${DSName}" --project "${ProjectName}" --replace --no-certcheck

                                     

                                    This leaves the metadata modelling intact, while updating the underlying extract.

                                    The great thing with this is that you can continue to make metadata model changes using the TDS file and re-publish without breaking the overall process (unless you start removing columns from TDE which other calculations depend on).

                                    1 2 Previous Next