Yes - it is set to 200 because I only wanted to create a small data extract for POC purpose (the original data source runs into approx 2bn rows)
Any workaround for this because if I edit the data source and set it to All rows, it will start fetching all the rows?
Ah I see - I'm fairly new to Tableau myself but I suspect you need this set to all rows initially as you are defining both the behaviour of the first extract and any subsequent appended files (i.e. limiting to 200 rows) in that first step. [Someone else please chip in if I am wrong!]
I guess if the POC is to test "can I append data" then the fact that it only takes 200 from each file isn't material if all you are trying to prove is that the concept works? Another suggestion would be can you get a reduced subset of your first file (just 200 rows) then extract that in full, before appending the two smaller files?
Thanks Michelle. I tried to extract a small subset by using filters and had selected "All rows" and it successfully appended all the rows from the child extracts.
However, what I learnt that:
- If you want to combine (or append) data from 2 different tde files, the data in the tde files has to be of exact same structure, same column order, name, type
- The most important thing - whatever filters you have set on the Master Extract, when you try to add the rows from the child extracts, the rows will be validated against ALL the filters / conditions present in the master extract. Only if it passes the conditions is when the rows are added to the master extract, or else the rows are ignored.