14 Replies Latest reply on Aug 7, 2013 8:08 AM by Toby Erkson

    Semantic layer in tableau

    Philip George

      Hi,

       

      Can any anyone give some details on how the semantic layer in tableau works.

       

      Regards,

      Philip

        • 1. Re: Semantic layer in tableau
          Russell Christopher

          Hey Philip -

           

          You've asked a pretty big question! Can you be a bit more specific?

          • 2. Re: Semantic layer in tableau
            Philip George

            Hi Russell,

             

            I have a basic understanding on the semantic layer provided by tableau.Please validate my understanding.Also if you have any docs on semantic layer of tableau.please share me that.

             

            In tableau Semantic layer helps to centrally manage the data sources,metadata,calculated fields etc.

            Data server is managing the semantic layer.

            So using this semantic layer users can connect to a single data source and work on the same data source for adhoc querying.

            This shared datasource is somethig similar to a universe in business object.

             

            Regards,

            Philip

            • 3. Re: Semantic layer in tableau
              Russell Christopher

              Yes to everything you said.

              2 of 2 people found this helpful
              • 4. Re: Semantic layer in tableau
                Toby Erkson

                The difference between a BO Universe -- or a Cognos Framework Model -- and Tableau's shared data source is the absence of rich metadata in Tableau.  For me and my end users that's not an issue!

                2 of 2 people found this helpful
                • 5. Re: Semantic layer in tableau
                  Stephen Pace

                  I have a related question.  Is there an API third party tools can create this semantic layer for Tableau without going through their interface?  Or is there plans for Tableau to expand participation with MITI to assist with this?  I notice there is an "export from" Tableau MITI bridge, but not "write to", which is what I need:

                   

                  Tableau MIMB Import Bridge

                   

                  For example, a data warehouse automation product like Kalido can create and manage a huge data warehouse, and because it created it, Kalido knows every table name and join in the system as well as the business definition for each object.  It can translate that information automatically to BI tools like Business Objects and Cognos via their semantic layers.  Is this feasible in Tableau?

                  1 of 1 people found this helpful
                  • 6. Re: Semantic layer in tableau
                    Russell Christopher

                    Hey Stephen -

                     

                    This is an interesting scenario- can you give me a feel for why this would be useful to you?

                     

                    To answer your question -- No. Tableau doesn't document this metadata layer, so we don't support anyone hacking it / automating it / etc. one way or the other. However, Tableau partner Interworks has a software/services offering to do this sort of thing. If you're planning on coming to TCC, they'll be showing it off.  (No, they don't sell it as an SDK-only type thing yet, you have to use services)

                    1 of 1 people found this helpful
                    • 7. Re: Semantic layer in tableau
                      Stephen Pace

                      Sure, I can give you a scenario.  Let's say I have a mature data warehouse with hundreds or thousands of entities managed by Kalido.  Kalido knows everything about the physical schema, how the tables join, multiple versions of time variance (using Kimball Hybrid SCD ideas), as well as the business definitions and friendly names for objects.  Kalido knows if there are loops in the model and if so, how to resolve them. Kalido also knows if measures are summable or not (for instance, if I have temperature as a measure and I want to see if my ice cream sales go up when the temperature goes up, I need to tell the BI tool not to sum temperature).  If you were to create a Business Objects Universe from scratch for a warehouse this size nicely formatted with all business definitions, it might take you three or four weeks to do right.  Contexts can be used for loop resolution, entities can be put in folders with friendly names, business definitions are made available for tool tips in the reports, etc.

                       

                      Because there is an API for the Business Objects semantic layer and they share information with MITI, Kalido can automatically build the complete Business Objects Universe in 5 minutes.  We can also do this for Cognos and for Microsoft BI for the same reason.  What this means in practice is that you can go from a new requirement that you put in our business model, to building the physical schemas in dev, to loading a bit of sample data, to building the BI semantic layer, and go back to the business user 10 minutes later showing them their result directly in their BI tool of choice.

                       

                      That is powerful--that is the foundation of what Agile BI should be.  But if the BI tool can't keep up with the speed of change in the physical layer of the data warehouse, the BI tool ceases to support Agile.  We do what we can for the next generation of tools.  We can generate a result set table and feed it directly to QlikView or Tableau or Spotfire.  For QlikView we have an option to automatically write all of their import scripts so all you have to do is format the dashboard (all data that you select will automatically be populated into the cube).  We'd like to do more, but if the BI tool has a closed metadata layer, there is only so much we can do.  In those cases I generally say that the end user is no worse off than if they were doing custom build (e.g. you have to use the BI tool wizards to manually build the lens on the data) but manual build is no match for automatic build, especially if the tradeoff is multiple weeks of development time in a project, or having to bring in a more expensive 'guru' resource rather than a cheaper resource that can just do the report formatting.

                       

                      Bottom line, if a BI tool has a closed metadata layer, it is very difficult to have that tool part of initial development phases, and very hard to recommend it if we are asked our opinion on what BI tools should be evaluated.  Tableau certainly has some very nice features, but semantic layer access is one place where you are behind.  Hopefully you will put some resources into it in the future.  Thanks!

                      1 of 1 people found this helpful
                      • 8. Re: Semantic layer in tableau
                        Toby Erkson

                         

                        Stephen Pace wrote:

                        ...

                        Bottom line, if a BI tool has a closed metadata layer, it is very difficult to have that tool part of initial development phases, and very hard to recommend it if we are asked our opinion on what BI tools should be evaluated.  Tableau certainly has some very nice features, but semantic layer access is one place where you are behind.

                        I completely disagree here.  This is a common statement I hear from people who are used to traditional, IT-centric reporting environments.  It's what they're used to dealing with so, of course, it makes sense as "that's how it's always been done" and in the past it was also the norm (at least in corporate situations).  There was a time when I believed this.  The power of Tableau is not truly realized here.

                        Stephen Pace wrote:

                        ...

                        We'd like to do more, but if the BI tool has a closed metadata layer, there is only so much we can do.  In those cases I generally say that the end user is no worse off than if they were doing custom build (e.g. you have to use the BI tool wizards to manually build the lens on the data) but manual build is no match for automatic build, especially if the tradeoff is multiple weeks of development time in a project, or having to bring in a more expensive 'guru' resource rather than a cheaper resource that can just do the report formatting.

                        This says a LOT to me, that many don't comprehend what Tableau is.  Tableau is for custom builds!  It is for playing with, experimenting with, and having the bonus of being able to turn it into a canned report.  "Automatic reports" fall so terribly short here.  They rely upon a lot of IT behind-the-scenes work (you're right, it can takes weeks/months).  And that's just it, this tool isn't designed for report formatting, it is for data exploration and visualization rapidly.  It doesn't require heavy IT interference to work; it gets people the information they need now.  If they don't use this tool then they'll find another, 99% of the time that tool is Excel.  And then -- sooner or later -- they hire ME (an expensive guru) as shadow IT to fix/improve/automate the process because IT is too slow to react and/or too busy to help.

                         

                        What IT management fails to get is that there is a need for multiple types of reporting tools, not just one.  A massive meta-data layer is necessary as it helps in understanding the data and finding that single version of the truth BUT it does not dictate what tools to use to turn the data into information.

                        • 9. Re: Semantic layer in tableau
                          Stephen Pace

                          Nowhere here do I mean anything to do with formatting reports or 'automatic' reports, whatever those are.  I'm talking about automated or wizard-driven semantic layer creation, work that needs to happen before a report can be created.

                           

                          Also, I never said that a semantic layer dictates what tools should be used. Kalido is BI tool agnostic. What I care about, is if I have 3000 tables all with German acronyms, and I need to make some of them available to the BI environment, what is the best way to do that?  Turn those tables over to you and say "have at it, figure it out yourself?"  Or, since I already know what the friendly names are, and already know how the tables are arranged (drill paths, etc.), can I help you by creating some or all of that layer for you?  My point is that any function you can do in your BI configuration environment manually like assigning a friendly name for an obscure column, providing a business definition for an object so that the end user can hover their cursor over an item to get more information, predefining drill paths, defining a measure to sum or not, etc. are all things that I should have the opportunity to create automatically if I already know them.  Also, if I can create it automatically, it means the consultants can spend less time on manual drudgery and more time providing useful analysis.

                           

                          Tableau (and a few other vendors, Tableau isn't the only offender here) currently suggest that having this be automated isn't that a big deal.  This is for a number of reasons, not the least of which is that they don't have the feature yet.  Sometimes it is because the bread and butter business is selling 50 seat departmental solutions instead of 1000 seat enterprise ones, and the volume and complexity of data is small enough that a 'guru' can power through it.  However, as Tableau and QlikView and Spotfire move upmarket into the realm of the larger established BI players and compete for the 1000 seat installs (I know they are winning some of these already), they are going to have to deal with this scenario more and more.  The other side of the scenario is pushing some of that BI metadata back into other environments--there is already a MITI beta read-only Tableau bridge, so clearly someone is already thinking about this.  Further still is the 'ongoing change' problem.  If someone renames a table that BI is using, do you have to manually go update every place that BI references it?  Or if you have API level access to the semantic layer, is it possible to rename that table in an automated way without breaking everything downstream?  In 1000 seat installs, changing things can be painful, often involving people in other geographies, so the more you can automate, the better off you'll be.

                           

                          You say "absence of rich metadata in Tableau" is "not an issue", but my guess is that when Tableau makes some improvements you'll be a fan of what they do.  You also mention that "a massive meta-data layer is necessary" but this misses my point as well.  I'm saying whatever the current capability of a tool (e.g. the bits you configure manually today), I should have the opportunity to create those same objects in an automated way.  That's it.  At that point, it is my responsibility to map what I know to the capabilities of that tool.  For some tools, that will be more than others.  But even if I can only build 80% of it, you'll still be better off than starting at 0%.

                          1 of 1 people found this helpful
                          • 10. Re: Semantic layer in tableau
                            Russell Christopher

                            Hey Stephen -

                             

                            Thanks for the feedback. I think the point that you make is valid, but (right now, anyway) "it's not about the metadata" for the average Tableau customer.

                             

                            What do I mean? Well, the BOBJ customer interested in Tableau that I speak to wants nothing to do with anything nearly complex as a Universe, Those who were reasonably successful with these "boil the ocean" metadata layers were successful in spite of them, not because of them. We co-exist with these solutions because we are faster, and easier to use. We're not trying to solve "their" problem.

                             

                            Self-reliance is one of the key reasons users like our stuff - and an Enterprise approach you seem focused on runs counter to that. There's a place for it, but it's more of an edge case scenario for us.

                            • 11. Re: Semantic layer in tableau
                              Stephen Pace

                              Sure, I get that, different tools for different purposes.  But I'm not talking about "boil-the-ocean" metadata.  I'm talking about just the metadata you configure manually in Tableau today.  Even if it is just the friendly name for a column in a database.  If you have 20 elements, I get it--you can type that in pretty quickly.  But if it is 500, I've saved you some typing.

                              1 of 1 people found this helpful
                              • 12. Re: Semantic layer in tableau
                                Russell Christopher

                                Yup, it would be nice. However, I'd (personally) rank something like that relatively low on the "to do" list if I were a  PM here - we get relatively few asks for that sort of stuff compared to other things.

                                • 13. Re: Semantic layer in tableau
                                  Stephen Pace

                                  Makes sense, always more to do and finite resources to do it.  Funny thing, as I was hitting save on my previous answer, this popped into my inbox:

                                   

                                  Design Tip #158 Making Sense of the Semantic Layer - Kimball Group

                                   

                                  Great timing.  :-)

                                  • 14. Re: Semantic layer in tableau
                                    Toby Erkson

                                    Stephen, thanks for the explanation.  I better understand where you're coming from