4 Replies Latest reply on Jan 17, 2017 1:13 AM by jens.bruckmann

    WDC increment refresh help needed

    jens.bruckmann

      Hi all

       

      I just found the article https://tableau.github.io/webdataconnector/docs/wdc_incremental_refresh with an example how to use it. Ok I think I understood the first part

       

      myConnector.getSchema = function (schemaCallback) {
          var cols = [
           { id : "id"           , alias : "Sensor ID", dataType : tableau.dataTypeEnum.string }, 
           { id : "datetime"     , alias : "DateTime", dataType : tableau.dataTypeEnum.datetime },
       { id : "coverage"     , alias : "Coverage", dataType : tableau.dataTypeEnum.string }
      
         
          ];
      
      
          var tableInfo = {
              id : "CM",
        alias : "Capacity Management",
              columns : cols,
        incrementColumnID: "datetime"
          };
      
      
      

       

      As I want to do the increment refresh by datetime I have added the line 15 where it is saying to increment the refresh by checking the datetime

       

      But how do I implement this in my table, donecallback?

      myConnector.getData = function(table, doneCallback) {
          var lastId = parseInt(table.incrementValue || -1);
      
      
          // Gather only the most recent data with an ID greather than 'lastId'
          // ......
      
      
          table.appendRows(dataArray);
          doneCallback();
      };
      

       

      is the part of the example. Is it the same when I need to check datetime instead of a rownumber?

        • 1. Re: WDC increment refresh help needed
          Patrick A Van Der Hyde

          Hello Jens Bruckmann

           

          I moved this thread to the Web Data Connector where someone might be able to assist. 

           

          Tamas Foldi

          • 2. Re: WDC increment refresh help needed
            Tamas Foldi

            Yes, you need two changes. The first what you did is to change incrementColumnID to datetime, second filter your results in getData using table.incrementValue that will have the latest datetime from your previous refresh. Of course you should not parse that value as integer

            • 3. Re: WDC increment refresh help needed
              jens.bruckmann

              Hi Tamas,

              thanks a lot for jumping in.

               

              Please let me explain what I need to get:

              I try to run a script which looks at different sensors and to get the data. The Problem what I am facing I can´t do a full refresh all time as the sensors have a timeout implemented not to provide any data, if it takes longer than 60 sec. So if the datafile is too large I am not able to get any data anymore. So I think I need to implement the incremental refresh. In the full script below I already have implemented a loop to run one Sensor after the other. So I need to bring the incremental refresh somehow into the loop, so look at sensor xy, check the latest entry in the extract update the Data and do the same with the next sensor till all sensors are worked out. In the link you find and a sdate and an edate. These are for StartDate and EndDate. The startdate is fixed at moment and the Enddate is a variable. sdate and edate are mandatory. Do I need to run the script once with a fixed sdate and then change the sdate to var (datetime???) or how can this be sorted?

               

              Maybe you know exactly what I need to change when you see the script

               

              (function () {
                  var myConnector = tableau.makeConnector();
              
              
              myConnector.getSchema = function (schemaCallback) {
                  var cols = [
                   { id : "id"           , alias : "Sensor ID", dataType : tableau.dataTypeEnum.string }, 
                   { id : "datetime"     , alias : "DateTime", dataType : tableau.dataTypeEnum.datetime },
                   { id : "Total"       , alias : "CPU Load", dataType : tableau.dataTypeEnum.int }
                 
                  ];
              
              
                  var tableInfo = {
                      id : "CM",
                      alias : "Capacity Management",
                      columns : cols,
                      incrementColumnID:"datetime"
                  };
              
              
                  schemaCallback([tableInfo]);
              };
              
              
                var tableData = [];
              var feat = [];
              
              myConnector.getData = function(table, doneCallback) {
              
                var ids = ["3388","63396","63397"];
                   date = new Date();
                  edate = date.toISOString().slice(0,10) + "-23-59-59";
                var lastdatetime =parseInt(table.incrementValue || -1);
                for (var i =0; i < ids.length; i++)
                {
              
                var id_temp  = ids[i];
                var api_req = "https://***.json?id="+id_temp+"&avg=0&sdate=2017-01-01-00-00-00&edate="+edate+"&username=tableau-srv&passhash=*****"
              // $.getJSON(api_req, function(resp) 
                $.ajax({url: api_req, dataType: 'json', async: false, success: function(resp){
              
                feat = resp.histdata;
              
              
                for ( var j = 0, len = feat.length; j < len; j++)
                {
                              tableData.push({
                "id" : id_temp,
                                  "datetime": feat[j].datetime,
                "Total": feat[j].Total
              
                              });
                          }
              }});
              
              
                };
                  table.appendRows(tableData);
                doneCallback();
              };
              
              
               tableau.registerConnector(myConnector);
                $(document).ready(function () {
                  $("#submitButton").click(function () {
                      tableau.connectionName = "Capacity Management";
                      tableau.submit();
                  });
              });
              })();
              

               

              any help really is appreciated

              • 4. Re: WDC increment refresh help needed
                jens.bruckmann

                Hi,

                 

                any help really appreciated    

                 

                Tamas Foldi ?

                 

                please?