10 Replies Latest reply on May 16, 2018 9:25 AM by poorni rajarajan

    Tableau SDK and Extract API 2.0 Incompatible?

    scott.steesy

      With all the hype over the new Hyper extract format I went looking and found the new "Extract API 2.0" download.

      It appears to be functionally used identically to the Extract API in the SDK, and barring a #define difference in the TableauCommon.h header file that part is identical.

      I'd like to be able to produce both/either TDE and Hyper files from the same application, and use the ServerAPI in the SDK to publish either to Tableau Server (or Online).

      BUT there appears to be is a major problem here. The API 2.0 was built using different versions of many of the (bin) DLLs than the SDK, so it appears you can't use both in the same application!

       

      Since the API 2.0 has a DLL named TableauHyperExtract.dll vs the SDK's TableauExtract.dll, and all the common code seems to be compatible, it certainly seems like Tableau could have easily just added Hyper to the SDK as say "HyperAPI" instead of keeping it separate.

      But, if they would just update the SDK to use the same version of the 3rd party DLLs and rebuild and release that, then we could merge the bin folder's contents and use both from the same application.

       

      Speaking of the bin folder ...

      The Extract API 2.0 download contains a bin folder with DLLs similar to the SDK, but then it has a "hyper" folder under that with several of the DLLs repeated and then the EXE files for hyper, similar to the TDEServer64.exe SDK has.

      Why the duplication? Would I need to lay it out like that on the machine where my app will run?

      Then to get more weird, there is a ZIP in the bin/hyper folder that nested 3 folder deep in it contains the DLLs again, as well as PDB files so appears to be for DEBUG, but there is no Read Me file to explain anything and the Online Help doesn't either.

        • 1. Re: Tableau SDK and Extract API 2.0 Incompatible?
          Patrick A Van Der Hyde

          Hello Scott,

           

          Thank you for your post.  I am investigating with the team at Tableau.

           

          Patrick

          • 2. Re: Tableau SDK and Extract API 2.0 Incompatible?
            James Chen

            Hi Scott,

             

            Thank you so much for trying out the new Extract API 2.0 and reaching out to us on the forums! I’ve broken up your post a bit into what I think your main questions/concerns are and addressed them separately.

             

            • The API 2.0 was built using different versions of many of the (bin) DLLs than the SDK, so it appears you can't use both in the same application!
              • This is by design – a single application cannot load both TableauExtract.dll and TableauHyperExtract.dll. The reasoning for this is having the TableauHyperExtract.dll export the same symbols as the TableauExtract.dll makes it simpler for customers to upgrade their Tableau SDK applications to the Extract API 2.0. With this implementation, the only thing that needs to be changed is including the new header and linking the new library.

             

            • If they would just update the SDK to use the same version of the 3rd party DLLs and rebuild and release that, then we could merge the bin folder's contents and use both from the same application.

             

            • The Extract API 2.0 download contains a bin folder with DLLs similar to the SDK, but then it has a "hyper" folder under that with several of the DLLs repeated and then the EXE files for hyper, similar to the TDEServer64.exe SDK has.
              • This is by design as well – Hyper is designed to run in its own binary directory. The DLLs are actually different – they are either different versions or built with different compilers/compiler settings. The “bin/hyper” directory comes with Tableau Desktop 10.5.

             

            • Would I need to lay it out like that on the machine where my app will run?
              • You do not need to modify the layout of anything inside the “bin” or “lib” in the extract API package.

             

            • There is a ZIP in the bin/hyper folder that nested 3 folder deep in it contains the DLLs again, as well as PDB files so appears to be for DEBUG, but there is no Read Me file to explain anything and the Online Help doesn't either.
              • We will address this in our next maintenance release.

             

            Thank you again for using the Extract API 2.0 and please let me know if you have any more questions!

             

            Best,

             

            James Chen

            jachen@tableau.com

            Associate Product Manager, Hyper Team

            • 3. Re: Tableau SDK and Extract API 2.0 Incompatible?
              scott.steesy

              Since I posted this question I have been doing a bit of experimenting.

              Your reply is wrong in several regards.

              1) Two DLLs with the same exports can be used in the same application as long as you alias the imports so they appear different to the caller. Microsoft does that all the time, which is how they can make both a 32 bit and a 64 bit API appear the same.

              2) It turns out the 3rd party DLLs that overlap between Extract API 2.0 and the SDK are fully backward compatible, as well as Tableau's own DLLS (like TableauCommon).

              3) I was unable to get the Extract API 2.0 to work at all until I placed everything in the bin folder and the bin\hyper folder into a single folder, either with my app or on the PATH. I suppose I could have put both folders on the PATH but that seems ridiculous. So for a production app to support the Extract API 2.0 not on the PATH, all the files do have to be in the program folder together.

               

              In the end I was able to get a single application to be able to use both the SDK and the Extract API 2.0 simultaneously. I had to put the EXTRACT API 2.0 files in a single folder list on the PATH first, so the newer files were found where-ever there was overlap, immediately followed by the folder holding the Tableau SDK. Then in my app, I aliased the imports from the HyperExtractAPI, and finally I abstracted all the "extract" classes so they had a common base class making it so all my code would work with either one. Then it was just a single line of code difference in my code to initialize whichever one (TDE or Hyper) I wanted at that moment. I have successfully created a TDE then a Hyper or vice-versa in a single run. I have yet to attempt producing one of each at the same time using different threads.

               

              I posted my C# combination of the SDK and the Extract API 2.0 as an update to my earlier C# work on the original Extract API at DataExtract API ported to Dot.Net (C#)

              • 4. Re: Tableau SDK and Extract API 2.0 Incompatible?
                scott.steesy

                Oh, I'm VERY disappointed to hear you are abandoning the Server API because it is so easy for we non-web developers to use versus having to hack out (probably fragile) code to use the REST API in a non-web app.

                At this time I can still use the Server API to publish TDE files, but it chokes on a Hyper file even if you rename it to have a TDE file extension. Interestingly, if you publish a TDE to Server 10.5 the server says it is in Hyper format even though it still has the TDE extension. And no matter what I tried in the calls to the Server API, it always got to the server with the ".tde" extension. If I could have got it not have an extension, or change it I would have been happy.

                • 5. Re: Tableau SDK and Extract API 2.0 Incompatible?
                  Steven Lockhart

                  Hi Scott,
                  This is Steven from Tableau Development. Thank you so much for developing with the next Extract API v2.0. I'd like to follow-up on some of your points in this thread.

                   

                  First, we're excited to hear that you found a way to load both the TableauExtract and TableauHyperExtract shared libraries in your application and we'd love to hear more about it.

                   

                  But second, your experience of needing to move around the contents of the unzipped Extract API v2.0 in order to get it to work should not be necessary. If we say "<install-dir>" is the location of an unzipped and unmodified Extract API v2.0 archive, then if "<install-dir>\bin" is added to your %PATH%, the Extract API's internals will handle the fact that the Hyper binaries are located in "<install-dir>\bin\hyper". You should not need to flatten the directory structure. For more reference, the online help docs go through the build configuration for a Visual Studio project for the C and C++ sample applications.

                   

                  Finally, regarding the deprecation of the Tableau Server API in the Tableau SDK: in addition to the REST API we also offer a Tableau Server Client Library (Python) which might better address your publishing needs.


                  Thanks again for reaching out to us about the Extract API,
                  Steven

                  • 6. Re: Tableau SDK and Extract API 2.0 Incompatible?
                    Steven Lockhart

                    Hi Scott,
                    This is Steven from Tableau Development. Thank you so much for developing with the next Extract API v2.0. I'd like to follow-up on some of your points in this thread.

                     

                    First, we're excited to hear that you found a way to load both the TableauExtract and TableauHyperExtract shared libraries in your application and we'd love to hear more about it.

                     

                    But second, your experience of needing to move around the contents of the unzipped Extract API v2.0 in order to get it to work should not be necessary. If we say "<install-dir>" is the location of an unzipped and unmodified Extract API v2.0 archive, then if "<install-dir>\bin" is added to your %PATH%, the Extract API's internals will handle the fact that the Hyper binaries are located in "<install-dir>\bin\hyper". You should not need to flatten the directory structure. For more reference, the online help docs go through the build configuration for a Visual Studio project for the C and C++ sample applications.

                     

                    Finally, regarding the deprecation of the Tableau Server API in the Tableau SDK: in addition to the REST API we also offer a Tableau Server Client Library (Python) which might better address your publishing needs.


                    Thanks again for reaching out to us about the Extract API,
                    Steven

                    • 7. Re: Tableau SDK and Extract API 2.0 Incompatible?
                      scott.steesy

                      Easy to load 2 DLLs with conflicting public function names.

                      Call LoadLibrary to load the DLLs and get a handle to each.

                      Then use GetProcAddress using the handle of each to get a function pointer to the specific DLL, and assign it to a properly typed variable.

                      Then call the methods using the function pointer variable, which having the correct signature makes everything easy to code and use.

                      I learned to do that in early days of DLL ****.

                      I modern languages, like C#, it is ridiculously easy since you define the signature of the method, and the DLL it exists in, in a single statement which code attributes and the framework takes care of doing everything said above.

                       

                      I have several apps and some are 32 bit and some 64 bit depending 3rd party drivers' bitness, so I couldn't have both SDK folders on the Windows System or User PATH because one or the other would fail. So all my apps read the correct path to the SDK form their config file and add that the (in memory) process's PATH. Works like a charm with the SDK, and since I'm now also adding the Extract API 2.0 (EA2) to the 64 bit apps, I just did the same. The app did find the DLLs in the bin folder, but when starting to use them, they didn't find the stuff in the bin\hyper folder. So I had to flatten the EA2 folders and then it all worked.

                      Also because some identical DLLs exist in both the bin and bin\hyper folders it just seems like a waste of space, especially when I go to package it up into an install.

                       

                      I didn't even consider the "Tableau Server Client Library (Python)" since I code in C/C++ and C#, not Python. I download it and see if what it contains is callable for me.

                      • 8. Re: Tableau SDK and Extract API 2.0 Incompatible?
                        scott.steesy

                        I downloaded the "Tableau Server Client Library (Python)" library, and nothing in there I can directly call. Having never seen Python code before it was interesting. I could even follow it enough that I think I could port it to C++ or C# except for the fact it relies on imports of code that must be part and parcel of Python to make all the web calls and get the results.

                        I'd still have to figure all that out. Plus this library is a 0.1 version I'm guessing it is in flux and I'm not up for trying to port a moving target.

                        • 9. Re: Tableau SDK and Extract API 2.0 Incompatible?
                          David Howell

                          The Tableau Server Client Python library ( TSC)  is just a wrapper around the REST API, mostly using the third party requests python library , it's not part of standard library but is defacto standard.

                           

                          You might be better off just referring to the REST documentation and using something like RestSharp from nuget.

                           

                          Funny you mentioned hyper files uploading to the server with .tde extensions, I noticed the same thing in TSC. I've actually found that uploading hyper files directly is not that useful because there's no tds metadata packaged with it. I'm having to construct (or modify) tds files, then stuff them and hyper files into tdsx (PKZIP) files to be uploaded, seems to work better but not fun.

                           

                          • 10. Re: Tableau SDK and Extract API 2.0 Incompatible?
                            poorni rajarajan

                            Hi Scott,

                             

                            Can you please provide DataExtract API ported to Dot.Net (C#) code base.

                            Can you please share the sample code for .hyper extract from c#.

                             

                            I am getting "unauthorized" error while downloading