11 Replies Latest reply on Aug 9, 2018 2:45 PM by Dan Santon

    Tableau Prep - Max # of tables to Union?

    Sandy Willbanks

      Hi,

      I have just started exploring Tableau Prep.  I have a database of reservation bookings that happens to be split out month by month.  I want to union together all months (all tables) from Jan 2017 until June 2018.  I successfully dragged and added the first 10 tables to the Union, but after that point, nothing happened.  Each time I try to drag month 2017-11 nothing happens.

      Is there a (10) table limit to a Union in Tableau Prep??

       

      Thank you,

      Sandy

        • 1. Re: Tableau Prep - Max # of tables to Union?
          Don Wise

          Hi Sandy,

          If there's limit on the number of tables I haven't found anything that confirms it.  Here's a link to the instructions for using the wildcard search to union all similar tables:

           

          • Matching Pattern (xxx*): Enter a wildcard search pattern to find files that have those characters in the file name. For example, if you enter ord* all files that include the file name are returned. Leave this field blank to include all of the files in the specified directory.

          To use wildcard search to union files:

          1. Click the Add connection button and under Connect, click Text File for .csv files or Microsoft Excel for Excel files, and then select a file to open.
          2. In the Input pane, select the Multiple Files tab, and then select Wildcard union.

           

          This might help in bypassing the limit if there is one? Not sure.  I didn't see anything in the online help that would indicate a limit on Union's: Join or Union Data Hope that helps!  Don

          • 2. Re: Tableau Prep - Max # of tables to Union?
            Sandy Willbanks

            Thanks for the response!  I am finding/guessing that the Wildcard union only works with Excel or .csv files.  I am using a mysql database and I do not have that option.

             

            Thanks!

            Sandy

            • 3. Re: Tableau Prep - Max # of tables to Union?
              Don Wise

              OK, well my only other thought, ugly as it might be, is to do what you're doing, then create a new Union group (Union 2) then Union group 1 to Union group 2 thereafter...

              • 4. Re: Tableau Prep - Max # of tables to Union?
                Sandy Willbanks

                Thanks!  Thought about that.... wasn't sure if it would work.. I think I will try it though.  It still beats having 20+ union statements in a custom query

                 

                Thanks!

                Sandy

                • 5. Re: Tableau Prep - Max # of tables to Union?
                  Isaac Kunen

                  Hi Sandy,

                   

                  Yes: there is a hard limit of 10 tables per union. This is mostly there because the UI doesn't scale very well past 10 (and it's stretching it even for 10). But you should be able to just create another union step to bring together more tables as Don suggested.

                   

                  Cheers,

                  -Isaac

                  • 6. Re: Tableau Prep - Max # of tables to Union?
                    Sandy Willbanks

                    Thanks Isaac!

                     

                    I sent this in to Tableau Support as well and got this reply:

                    " There really is not a limit as to the number of tables you can add in a

                    union, I was able to add 15 tables when testing. "

                     

                    Interesting...

                     

                    However, for me, I cannot add more than 10 tables in a Union.  I did that

                    and added a 2nd Union with an additional 5 tables, but had no luck.  It

                    tried to run the flow for 24+ hours... basically timed out.  For me I think

                    this is an issue with the data source itself, but was hoping that setting

                    up the flow in Prep would help as opposed to doing the Unions in a Custom

                    SQL Query like we are doing now.

                     

                    Thanks for everyone's help

                     

                    On Mon, Jun 11, 2018 at 10:18 AM, Isaac Kunen <tableaucommunity@tableau.com>

                    • 7. Re: Tableau Prep - Max # of tables to Union?
                      Isaac Kunen

                      Hi Sandy,

                       

                      There may be some confusion between a union in the flow, and the input union (aka wildcard union). The input union has no hard limit, but won't work for database tables -- yet.

                       

                      If you're having trouble with chained unions, we definitely should get an item filed through support. That should work.

                       

                      -Isaac

                      • 8. Re: Tableau Prep - Max # of tables to Union?
                        Barata Ribs

                        Hi Sandy, I faced the same trouble, I´m also loading the tables from a MySQL database and after the tenth table it doesn´t allow me to add more tables to the union.

                         

                        Did you find a solution to your problem?

                         

                        (As you probably noticed, no problem creating the union in Tableau Desktop)

                        • 9. Re: Tableau Prep - Max # of tables to Union?
                          Dan Santon

                          I'm also having an issue with Wildcard Input Unions in Tableau. I've got 66K+ CSV files in a local folder 329 columns wide, about 140ish rows each, ends up being about 37 GB. Tableau Prep churns and churns but it runs out of memory. Is there a way to chain union through prep?

                          • 10. Re: Tableau Prep - Max # of tables to Union?
                            Barata Ribs

                            Hi Dan, I think theoretically you could union 10 csv files, than union another 10 files, and union these two unions to make it a 20 files union.

                             

                            If you 65k+ files this is obviously not a solution, this is why I said theoretically.

                             

                            But let me ask you something, you managed to start your union step? I mean, all you 65k files were added to your union, but Tableau Prep is having a hard time to process all that info?

                            • 11. Re: Tableau Prep - Max # of tables to Union?
                              Dan Santon

                              I'm loading all of the files using a Wildcard Union in the Input step. The Tableau Prep steps are just to load all the csvs in the input step and output a hyper extract. That runs out of memory after 2 hours of churning.

                               

                              I've since pared the dataset to 459 files in the hopes I can develop the flow to eventually run on the 65K files. It's really slow to load in the field selection section of the input step too.