I have to create an extract starting from txt files.
I have 12 different txt files that represent my fact tables and which need to be appended in a union (directly in Tableau from version 9.3 ) . In addition, I have other three txt files that I add through left join to the huge fact table I created through the union.
Just after I have also to pivot some field of the fact table since I need a long format instead of wide.
Well, all these operation must be repeated each month since the raw txt data get updated.
The problem here is the CREATING EXTRACT loading time. (consider that the resulting extract is around 100 millions rows)
- Sometimes it takes just few minutes to start to import the rows and just normal time to finish them.
- Some other times it just stops at "creating extract database" stage forever and it seems to stop there as in the picture. (I tried to wait but after 800 mins of waiting I closed Tableau)
Do you guys have any idea of what is causing this random time waiting? Any suggestion about increasing the efficiency of this process?
Thanks a lot!