2 Replies Latest reply on Sep 6, 2016 6:28 AM by Bryan Haynie

    Extracts dissapearing when I click "Data Source" tab

    Bryan Haynie

      I've used the best method I can think of to create a data source for some dashboards but I'm thinking it may be causing me some issues.  I've done it this way because we do not have our data anywhere on a server we can query, due to the the back end work that needs to be done to clean up the data after it is pulled from the DB.  Also the data is much too large to put into one excel or access file.  (around 500k rows per month)

       

           1. I created an extract for month Jan15 of data

           2. I appended this extract with monthly files of data all attached to this "Jan15" file, through Jul16. 

           3. I created charts and views with this extract

       

      My first issue is this: When I click back to the "Data Source" tab, Tableau refreshes my data source, in effect deleting all of the extracts from my sheets I've created.  What is the solution to this?

       

      My second issue is, do any of you have any suggestions as to a better way to keep the data so that I can add the Month to date data?  It is my understanding that If I have new data weekly but sometimes the published data will change during the month, I need to be able to refresh to the most recent version of the most recent month's data.  For example, I need to add August data now but I know that If I create an appended extract, I will not be able to delete this and then re-append the newest fiscal data next week, if that makes sense.

       

      I am also having an issue with a calculated field that will not let me do a simple calculation with data tied to a second source, but maybe that will wait until I see what's going on with the rest of these issues first.

       

      Thank you for any help with this!

        • 1. Re: Extracts dissapearing when I click "Data Source" tab
          Kent Sloan

          Hi Bryan,

           

          I would use a union to stack monthly .csv files before creating an extract. The downside with this is that you have to go through and add the new months file every month.

          Combine Your Data with Union in Tableau 9.3 | Tableau Software

           

          A in between solution is to use the command line to merge monthly csv files which would allow you to dump the files into a folder and merge them each month with a basic script which I have found to be easier when dealing with hundreds of files. Below is a basic summary that I found online of how this works.

          http://webpierat.com/2011/05/23/merging-csv-files-using-the-command-line/

           

          Long term I would really question why the data can not be kept in a single historical table on the data base and cleaned. If this is not considered possible you have much bigger problems with outdated data base infrastructure or unqualified database admins. A script to pull in the most recent months data and clean it on a schedule followed by the tableau extract refreshing on a schedule (if you have Tableau server) would be ideal. This would give you a system that updates automatically without messing with monthly files.

           

          Thanks,

          Kent Sloan

          InterWorks

          • 2. Re: Extracts dissapearing when I click "Data Source" tab
            Bryan Haynie

            Thanks Kent.

             

            I've upgraded to 9.3 last week and see that I can now use the built-in union feature to join .CSV files together in the Data Source screen.  This will work perfectly for my purposes however I'm having an issue with the date field for the data I'm working with.  Every record in the files have a date for the transaction which comes in as "Number(Whole)".  When I try to build with this, I get strange values that don't make sense so I try to change them to "Date" which in turn brings me "Null" for every record's date.

             

            I was reading up on this and it seems that I should be able to use the DATEPARSE( to correct this but when I use that I still am getting nonsense values that do not appear as the dates from my original files.  Any further suggestions on that piece?

             

            Thanks!