5 Replies Latest reply on Mar 1, 2018 12:33 PM by Nathan Secrist

    Tableau Server Extract Optimization

    Nathan Secrist

      I am having some trouble with an extract taking around 30 minutes and I need this extract to update every 15 minutes with the new data.


      Some caveats:


      1. The 'DateTime' field in my Microsoft SQL Database is a string, in the format of '2018-02-26T14:38:43.997-0800'
        1. This being a Microsoft SQL Database does not allow me to use "DATEPARSE"
      2. For this extract I only want to have the last 24 hours of data available
        1. I have been unable to figure out how to do an incremental extract without having a proper date field.


      What I've done:


      1. I've created an extract data source that uses some string splits to create a properly formatted DateTime for Tableau to use and filter to the last 24 hours.
        1. Some problems with this are that this table I am extracting from is growing at a rate of around 1,008,000 records a day. (1 Record per sample per minute)
          1. I believe that tableau is needing to extract the entire table to perform the string splits and format my proper DateTime column before filtering for just the last 24 hours.
      2. Currently this extract is not incremental, for the same reason as above... the work needing to be done would be on the same order and I don't think (correct me if I am wrong) I would see any gain in performance.


      What I am looking for:


      1. Some help in understanding the most efficient way to convert from a string DateTime in the format '2018-02-26T14:38:43.997-0800' to a Tableau acceptable DateTime that can be used to filter.
      2. I am debating on whether I need to update the Database and insert a proper DateTimeOffset (not supported by Tableau) or some other form of a DateTime.
        1. I am even considering breaking up the DateTime record into a Date and a Time... which I think might help the filtering and allow for a performant incremental refresh.