4 Replies Latest reply on Sep 6, 2017 2:46 PM by Oshri Maidanik

    Wildcard Union auto-update?

    mark.welsh

      Let's say I've got a wildcard union to union all txt files that begin with "Sales" in a directory.  When new files are added to the directory, how can I ensure those are added to my dataset?  Is the answer different depending on if it's live vs. extract? 

        • 1. Re: Wildcard Union auto-update?
          mark.welsh

          It's probably the same for a non wildcard... Trying to find a way to make sure that new text files are added to the union when they are added to the folder.  I could just refresh the extract once a day, but I don't want to have to re-run the entire union everyday. 

           

          Will Tableau scan the folder and only add new files to the union or will it re-run the entire thing - if I refresh the extract?

          • 2. Re: Wildcard Union auto-update?
            Patrick A Van Der Hyde

            Hello Mark, 

             

            I tested this out since I use wildcard with a bunch of google analytics files that I collect monthly. 

             

            If you have an extract, a refresh of the extract (just right click on the extract and select 'refresh data' ) and it will pull in the new files. This was very speedy in my test but looking at a performance recording and test where I removed files from the directory, I believe the query for the data source is doing a full refresh from the files in the folder on an extract refresh. 

             

            If a live connection is utilized, then Tableau will utilize any new files placed into the wildcard file location on restart or by using a refresh of the data source. 

             

            If you prefer to just add one file at a time, I suggest using the 'append data from a file' feature from the data extract menu. 

             

            I hope that helps you out. 

             

            Patrick

            • 3. Re: Wildcard Union auto-update?
              mark.welsh

              Hey Patrick,

               

              I tried it this morning and it looks like it did a full refresh.  It's doable, but takes a little longer than I would like.  I do wonder if I could utilize incremental refresh, but I don't know much about that.  Does it scan through one field and look for updates to it?

               

              Thanks for the help

              • 4. Re: Wildcard Union auto-update?
                Oshri Maidanik

                Hi Guys,

                Have figured this one out?@

                I also have a daily auto-added CSV file to a specific folder, with a naming convention to be picked up by the wildcard feature.

                Does it make a difference for live vs extract?

                Does it make a difference if the folder is Google drive / local / network?

                Does it make a difference tableau DT or server workaround?

                Many thanks!