4 Replies Latest reply on Jul 19, 2018 12:24 AM by Gary Cantigneaux

    Different source file with new data

    Gary Cantigneaux

           Dear All,


      New to Tableau and I start to see the advantage of this tool over the most common excel reporting solution we currently use.


      I have a small question but i cannot find a way of doing this in Tableau or at least I cannot find the best way.


      Currently we are receiving a CSV file every other day and this contains the latest data but we want to build up some  history of the data. In the CSV file received i added a "timestamp" field which is the day of the data.


      What I am trying to figure out is how to best integrate it, either by adding all of those data into the Tableau file (if yes, how) or whether it is best to use some sort of database?


      Thanks in advance for your help.



        • 1. Re: Different source file with new data
          Jim Dehner

          Good morning

          If I understand your question you get a csv file on a regular basis that contains the most recent incremental data you want to append to an existing data to form a complete data set.  There is a date field in the data with the upload date?


          yu could use a wildcard union (see the link) keeping the csv files in a single folder - the new data will be appended to the dataset and a field will be added with the path(file) name giving you a complete data set - yo can use the date field to filter for historic views


          Union Your Data




          If this posts assists in resolving the question, please mark it helpful or as the 'correct answer' if it resolves the question. This will help other users find the same answer/resolution.  Thank you.

          • 2. Re: Different source file with new data
            Gary Cantigneaux

            Thanks Jim, this helped me a lot, but i find that the result is a bit hard to understand.


            Let me explain:

            • I have created in test 4 files with different timestamp from the 15/7 to the 19/7. the Column are TimeStamp, Product, Week, Quantity
            • I then added some data to those files and created the union with the wildcard. => This worked great and gather new files automatically when i click the refresh button
            • I created a full Outer union (right being the copy of the files) that I linked in two columns (Product and Week) => I see all my data
            • Now I have the problem is that all my columns are duplicated (once for the "MasterFile" and one for the unions)

            This can create a problem for the end user if they select the wrong column. Is there a solution for this?





            • 3. Re: Different source file with new data
              Jim Dehner

              Good morning Gary


              I may not have understood the need here - I thought your process was that you receive a file every other day the represents the data accumulated over that 2 day period.

              The 2 day file needed to be appended to existing data to form a complete set of data for your analysis - the data in the master data set and the incremental 2 file have the same dimensions and measures - so you just appending the new data to the bottom of the existing data set


              The wild card unio will do that -


              see the example




              this is what the data file looks like in Tableau



              If you need to Join with other files ater you bring in the update I would suggest you look at Tableau Prep Tableau Prep



              It is a tool designed to shape, clean and connect data



              • 4. Re: Different source file with new data
                Gary Cantigneaux

                Thanks a lot Jim,


                I will have another look at your examples and get back to you.