Web Data Connector (WDC) for Webtrends

Version 3

    Hi all!

     

    After a fair amount of work I finished my WDC to a state where it does precisely what I want it to do. Since I'm not planning any further development (even though there is a lot you could add!) I thought I could as well share it with you...

     

    What is Webtrends?

     

    Webtrends is a web tracking service and they have a REST API. You can learn more here:

     

    http://www.webtrends.com/

    Data Extraction API

     

     

    Overview: How does the WDC work

     

    Just like other WDCs it's a pretty straight forward little Javascript (attached) doing an AJAX call to the API. Since the API is not open (like others) it requires an intermediate step: a CORS proxy (I'll leave it to Google to tell you what that is if you never heard it before).

     

    Since we live in a time where node.js is available it's really not that much of a big deal to get a CORS proxy up and running, even for free. I use Heroku and Rob--W's "cors-anywhere" module for this, but there are certainly other possibilities:

     

    Heroku | Cloud Application Platform

    Rob--W/cors-anywhere · GitHub

     

    Also, you will recognize there are some setTimeout functions in my code. This is just to avoid any conflicts with calling multiple tableau functions simultaniously. Want to change that? Go ahead! Generally, if you see something stupid in the code, feel free to comment here, I'm happy to learn

     

    So, what happens when you run the WDC in Tableau?

     

    Tableau opens the html file in its internal browser

    --> After preparing all the necessary fields, the html sends an ajax call to the CORS proxy

    --> The proxy adds some headers to the request and forwards it to the Webtrends API

    --> The API queries the requested data and returns a JSON

    --> The proxy forwards the JSON to the html (actually, to the internal Tableau browser, but you know what I mean)

    --> The html reformats the JSON into a readable table  and calls the Tableau magic to run ("tableau.dataCallback(...)")

    --> Tableau reads the formatted table and creates an extract / attaches the data to an existing extract

     

     

    More details: What do I have to do?

     

    See this WDC as a wire frame to develope your own WDC: You should know a little bit about Javascript and it doesn't hurt to go though the WDC Tutorial (http://onlinehelp.tableau.com/current/api/wdc/en-us/help.htm) to get a basic understanding of what it is doing.

     

    I didn't add any interface for user input, everything is hard-coded (even the credentials!) and therefore should never be publicly avaiable anywhere if you add your own credentials!

     

    So, what you need is this:

    • Set up a http host to run the WDC (you learn how to do this in the mentioned tutorial)
    • Set up a CORS proxy on Heroku (it's free when not used too extensively and there are tutorials for it, too)
    • Modify the html file to contain your credentials and API connection details, your CORS proxy... If you want to modify the API call URL, do this in the function "buildURL"
    • Adjust the way your Webtrends report needs to be transformed in the function "myConnector.getTableData" within the ajax call (I also hardcoded this part since in my report I had 3 dimensions plus the date column and I just wanted to avoid to set up the logic to parse the JSON accordingly... If your report is more simple than that it should be fairly easy to adjust the loops between line 120 and 137)
    • If you finished the WDC tutorial before you will realize there are also headers to be adjusted in the function "myConnector.getColumnHeaders"
    • If you don't want to use the date column for incremental refreshes (or don't have any date column in your report), you probably want to change the function "myConnector.init"

     

     

    Why no interface?

     

    The reason is pretty simple: I only use this WDC for one particular project and I didn't want to deal with the adjustments that would be necessary to enable Tableau to automatically refresh my extracts. So I simply avoided all the Tableau-authentication hassle and Tableau doesn't have to store any credentials. The WDC will only run locally on my laptop or a pretty much isolated server anyway, so I skipped this part

     

     

    This is waaay too much for me...

     

    Well, I'm actually expecting quite some work from you, that's correct. The reason for this: I simply don't have the time to set up the WDC as if it was a commercial product. If you do not want to deal with all that stuff, I know of one company that commercially released a Tableau WDC for Webtrends. It's much more comfortable to go with their solution Also, who knows, maybe Webtrends will release a WDC on their own one day?! Until then, feel free to play around with my solution and let me know here if you run into any problems. I might not be able to solve them all, but I will have a look

     

     

    Kind regards, Martin