2 Replies Latest reply on Mar 6, 2014 10:49 AM by Frank Yoshida

    Incomplete Tableau Extract

    Frank Yoshida

      Recently, we have been having problems with the refreshing of our data sources and data extracts.

       

      A nightly tabcmd refresh of the datasources which had worked for weeks previously, started to produce incomplete data. 

      A manual refresh from inside desktop produces the same result (in one case, only 28K of 112K shows up, in another 900K of 1.5M)

       

      The original datasources are all CSVs and if we start from scratch (connect to data from inside Desktop), it seems to work perfectly fine.

      Attempting to try different data sets with fewer rows/columns, but would eventually like a solution for the larger data set.

       

       

      Has anyone had similar issues?

      Is this a limit of rows/columns/cells that we are bumping up against?

       

      There seem to be previous similar issues, but those solutions do no seems to be effective for me...

      incomplete data extract

      Re: partial data import on extract refresh

        • 1. Re: Incomplete Tableau Extract
          Ralf Becher

          Maybe some dirt inside the data? Encoding, quoting or structural problems?

           

          - Ralf

          • 2. Re: Incomplete Tableau Extract
            Frank Yoshida

            Not exactly sure about the encoding, but they're CSVs that have been outputed normally.

            Opening the files (and playing with the data) in Excel doesn't show any issue.

            The one that's 112K rows has 39 columns, the one with 1.5M rows has 26 columns.

             

            A suggestion here about "move lines around in the file", got me to try sorting the data differently (Descending instead of Ascending) and that may have fixed it, but it'll take some new data to truly confirm.

            Losing rows when creating an extract from a large CSV file

             

            [UPDATE: Nope.  Sorting changes which records are included in the extract, but it's still an incomplete set. Sorting by date did allow me to see that the records are being loaded chronologically and where the records stopped.  There's nothing odd about that specific row that would make me think it is causing an issue]

             

            I suppose this suggests that it's an issue on the data side of things...