1 Reply Latest reply on Nov 27, 2013 4:07 PM by Russell Christopher

    Extract Limit Number of Fields

    Robert Royer

      I have built several 'General Purpose' extracts. These extracts contain basically all of the from a given data system. Most of our systems contain up to a few dozen fields once pulled into a star schema reporting database. We have one data source (our main customer billing system) that contains well over 100 data fields. Once we add reference tables and other related fields, this grows to over 400 fields. I have had great luck building extracts that are very tall (ie, lots of records, 50M+).


      Trying to create a 'General Purpose' extract on this massively wide data source fails every time. I am trying to give our end users a standard starting place that is not a direct connect into our SQL server (which is fairly slow when they start to build complicated calculations on top of the existing data). I know (and have built) custom extracts for specific workbooks that are performing slowly.


      However, I would like to avoid this when possible as it is time consuming on the front end, requires the user know all the fields they need up front (rarely happens), and gets a be a hassle to manage once published.


      Any ideas or pointers?