1 Reply Latest reply on Oct 19, 2015 1:45 AM by Cristian Vasile

    Web data connector scalability?

    Jonas Bergstrom

      Hi,

       

      What are your experiences using the web data connector to large data sources?

       

      Our company is planning to use web data connector to connect Tableau to a non-SQL data source containing a fairly large number of entries (order of magnitude 10-100 million entries).

       

      The data source interface is a RESTful web interface delivering data via JSON payloads. Typically each record contains 30-40 dimensions and 1 measure.

       

      With our PoC (basically a 20-line jquery javascript calling the url and directly passing the JSON result to Tableau) we seem to be hitting limitations at around 200K records as all of the data set is stored in memory.

       

      In your experience, is the web data connector scalable to this volume of data? What is best practice?

       

      If WDC is not scalable at this time, is the intention to develop it in this direction or the aim is to target "smaller" data sets with WDC?

       

      Best regards

       

      Jonas