You may want to use a tool like Alteryx to read data from the Tableau server and then integrate it with other DBMS.
When you say "hosts data sources on the server" do you mean TDE or hyper files ?
It sounds like you're trying to use Tableau as an ETL/ELT tool and it just won't work like that.
Tableau doesn't integrate the way you want, because it's never been designed that way. Tableau is usually "the end of the line" and not "the process used to take data out of A and put it in B"
Alteryx and other ETL/ELT tools will help, but nothing will help if you're only option is to read the TDE/hyper files on the server
I'm glad you included the last line about a direct connection not being an option, because unfortunately that IS the option that you should be going for. There's a difference between "go to Tableau for the data" when you are creating a few dashboards (yes, use Tableau for that), but "go to Tableau for the data" when you want to pull the gigabytes into another database is not the right option. Tableau will have problems displaying the data in the tabular form you're after, even before you download (CSV or XLS .... a 10GB XLS file ) and attempt to upload again.
Yes it is a hyper. But to clarify, I am using tableau desktop to connect to tableau server, where the data is hosted. I can also navigate to the server via browser and download the data source, which downloads as a (.hyper). So I assume that the the version I connect to thru desktop is also saved as hyper.
So unfortunately your "best"/"most reliable" option is the one you removed at the start - a direct connection to their DBMS. Also, no other tool opens a hyper database except Tableau, this will most likely change over the next year but it's not an option at the moment. This also means Alteryx - Alteryx can write to a hyper file but can't read from it.
Your investigation is now - how is the hyper file created ? It could be a simple Tableau extract refresh schedule which means that someone has the credentials to give you direct access to the DBMS or the hyper file is created another way (Alteryx maybe ?) and placed on the server so they would have the credentials.
Again, if you're exporting a few rows/columns from Tableau into another database then you might be able to use Tableau, when you start talking about 2-10GB then Tableau is not an option, it won't work (and if it does it won't work reliably and will take far too much memory) because it's not meant to be a data extract tool and it will fail (either now when you're building it or in the future when the data volumes are larger than today).
I've been through this situation a few times which is why I'm so strong in my opinion - please don't use Tableau as a data extract tool, it won't end well.