3 Replies Latest reply on Jul 22, 2016 3:23 PM by Dmitry Chirkov

    Collating lots (40+) of big excel files (500K+ rows) into a single source…

    Michael Carpenter

      I have some (currently 8, but eventually 40) big Excel files I need to collate data from. Each file has...

       

      • One big main data sheet (with 500,000+ rows, called DATA , which is the data I’m interested in)
      • Two other sheets with some “look-up” information (which I currently join to the DATA sheet on the Tableau import screen)

       

      I currently import and extract one file, then use “Data/Extract/Append from file…” to append the rest one at a time. This takes a long time (3-8 mins per file) and is becoming a pain as the number of files increases.

       

      Each excel file is around 50MB so I have to keep them separately (so I don’t think the new ‘union’ facility will help).

       

      I hoping there’s a better way to do this – any thoughts..?

       

      Also – am I going to run into problems as my number of import files increases (the extract is currently about 120MB)

       

      Thanks

       

      Michael