1 Reply Latest reply on Apr 11, 2016 12:52 PM by Brendan Lee

    Incremental Extract Column

    Gray Jones

      I am building a web connector to interface with our internal rest api.  My connector presents a gui to the user where they can select the service endpoint to use and after that which columns, filters, etc to choose from that endpoint.

       

      So I do not know which column to use as the incremental extract column until after they have made those choices.  But it seems that I have to specify the column to use in the init method which is before that has happened.

       

      Can I get some clarification on exactly when the incremental column has to be set?  Or when is the latest time that it can be set?

       

      And also I'm assuming that if I don't set the extract column then any subsequent refreshes will pull the entire extract?

       

      Thanks,

       

      Gray

        • 1. Re: Incremental Extract Column
          Brendan Lee

          Hey Gray,

           

          This is a docs bug.  The docs say you must set it during init, but you can actually set this property at anytime during the Interactive phase of the web data connector. For example, I took the IncrementalExtractConnector from the WDC SDK samples and moved the line that sets the columnId out of init.  The connector still worked fine. 

           

          Let me know if that doesn't work for you. I'll follow up with the team here to get the docs fixed, thanks for reporting!

           

          And yes, you are correct that if you do not set this attribute, all refreshes will be full refreshes.  With the column set, you are given the choice between a full and an incremental refresh when you go to refresh the data source.

           

          -Brendan