4 Replies Latest reply on Aug 15, 2016 6:30 AM by Sagar Agarwal

    Append Data from Data source not working properly

    Sagar Agarwal

      Hi All,

       

      I am doing a small POC on merging TDE files, and I am using the "Append Data from Data Source" feature in Tableau. However, I see that it is not working correctly. Here is what I see:

      3 TDE files:

      Master Extract - 200 rows

      Child Extract 1 - 300 rows

      Child Extract 2 - 500 rows

      I intended to add the rows of both Child Extracts into Master Extract, giving me total of 1000 rows. However, every time i select a Child Data Extract, it only adds 200 rows from it; not all the 300 & 500 rows. Screenshots below. Am I missing any tip / step here?

       

       

        • 1. Re: Append Data from Data source not working properly
          Michelle Hotchkiss

          Hi Sagar,

           

          When you go back to the main "Connect to Data" screen is your extract set to just bring in a subset of the appended data, i.e. just the top 200 rows? This should be set to All rows.

          • 2. Re: Append Data from Data source not working properly
            Sagar Agarwal

            Hi Michelle,

             

            Yes - it is set to 200 because I only wanted to create a small data extract for POC purpose (the original data source runs into approx 2bn rows)

             

            Any workaround for this because if I edit the data source and set it to All rows, it will start fetching all the rows?

            • 3. Re: Append Data from Data source not working properly
              Michelle Hotchkiss

              Hi Sagar,

               

              Ah I see - I'm fairly new to Tableau myself but I suspect you need this set to all rows initially as you are defining both the behaviour of the first extract and any subsequent appended files (i.e. limiting to 200 rows) in that first step. [Someone else please chip in if I am wrong!]

               

              I guess if the POC is to test "can I append data" then the fact that it only takes 200 from each file isn't material if all you are trying to prove is that the concept works? Another suggestion would be can you get a reduced subset of your first file (just 200 rows) then extract that in full, before appending the two smaller files?

               

              Michelle

              • 4. Re: Append Data from Data source not working properly
                Sagar Agarwal

                Thanks Michelle. I tried to extract a small subset by using filters and had selected "All rows" and it successfully appended all the rows from the child extracts.

                However, what I learnt that:

                - If you want to combine (or append) data from 2 different tde files, the data in the tde files has to be of exact same structure, same column order, name, type

                - The most important thing - whatever filters you have set on the Master Extract, when you try to add the rows from the child extracts, the rows will be validated against ALL the filters / conditions present in the master extract. Only if it passes the conditions is when the rows are added to the master extract, or else the rows are ignored.

                 

                Thanks.