If I understand your question you get a csv file on a regular basis that contains the most recent incremental data you want to append to an existing data to form a complete data set. There is a date field in the data with the upload date?
yu could use a wildcard union (see the link) keeping the csv files in a single folder - the new data will be appended to the dataset and a field will be added with the path(file) name giving you a complete data set - yo can use the date field to filter for historic views
If this posts assists in resolving the question, please mark it helpful or as the 'correct answer' if it resolves the question. This will help other users find the same answer/resolution. Thank you.
Thanks Jim, this helped me a lot, but i find that the result is a bit hard to understand.
Let me explain:
- I have created in test 4 files with different timestamp from the 15/7 to the 19/7. the Column are TimeStamp, Product, Week, Quantity
- I then added some data to those files and created the union with the wildcard. => This worked great and gather new files automatically when i click the refresh button
- I created a full Outer union (right being the copy of the files) that I linked in two columns (Product and Week) => I see all my data
- Now I have the problem is that all my columns are duplicated (once for the "MasterFile" and one for the unions)
This can create a problem for the end user if they select the wrong column. Is there a solution for this?
Good morning Gary
I may not have understood the need here - I thought your process was that you receive a file every other day the represents the data accumulated over that 2 day period.
The 2 day file needed to be appended to existing data to form a complete set of data for your analysis - the data in the master data set and the incremental 2 file have the same dimensions and measures - so you just appending the new data to the bottom of the existing data set
The wild card unio will do that -
see the example
this is what the data file looks like in Tableau
If you need to Join with other files ater you bring in the update I would suggest you look at Tableau Prep Tableau Prep
It is a tool designed to shape, clean and connect data
Thanks a lot Jim,
I will have another look at your examples and get back to you.