7 Replies Latest reply on Mar 9, 2012 11:48 AM by Shawn Wallwork

    Scoring...

    Shawn Wallwork

      Tableau Website? Forums? Whatever...

      Your new incarnation is excellent.

      OK, a few glitches, but overall a great improvement!

       

      But... (of course there's a 'but')

      I think you need to adjust your scoring system. The whole 'Top Participants' thing is great fun, but you haven't factored in either the quality of the answer, or the difficulty of the investigation required to get to the 'correct answer'. For instance, because I'm a relative 'newbie' (and I have a soft spot for helping my fellow newbies) I've climbed your 'Top Participant' ladder answering questions like this:

       

      http://community.tableau.com/thread/116688

       

      So I get a 4-point 'correct answer'. Really? This was certainly not worth a guru's time; yet I'm awarded max points. These sorts of answers don't come close to anything Richard or Joe usually step in and tackle. And they shouldn't be awarded 'premiumn points'.

       

      Tableau, please develope a scoring system that has a 'difficulty' factor attached -- so that newbie stuff (what I do) isn't confused with quality stuff (what Richard & Joe, and others do). I never want to (nor should I) pass Richard on any Tableau point system...

       

      Even if it's just for fun!

       

      Thanks,

      --Shawn

        • 1. Re: Scoring...
          Dimitri.B

          I agree with with the concern raised and all points are valid, but (of course there is always a 'but') - how would this new proposed system operate? It sounds like it must involve some sort of moderation, or human judgement on how difficult the question is.

          1 of 1 people found this helpful
          • 2. Re: Scoring...
            Shawn Wallwork

            Dimitri, what do I know? I'm definitely NOT the mathematician in this group -- 'but' maybe some sort of algorithm?

             

            --Shawn (soon to assume an alias )

            • 3. Re: Scoring...
              Dimitri.B

              I am no mathematician either, but I can't see how maths alone can handle this. There must be some numeric input into an equation or model, and what that input might be? The length of the post? Average word length? Frequency of certain words? How long the question remained unanswered? The list goes on...

              I think we cannot avoid direct human involvement here, and then rating difficulty is a subjective matter.

               

              Maybe have a system where anyone reading the question can rate its difficulty on some scale, and then assign different weight to those ratings based on user's personal score. In other words, if Joe rates a question as difficult, then his 3,000+ points carry a lot more weight than someone's 10 points. The system will then calculate the weighted average and points for the answer will be adjusted according to the difficulty.

               

              Then you have to consider things like:

              • was the question properly asked in the first place
              • did it have attached workbooks, screenshots, etc. if those were needed
              • who would bother reading and rating the difficulty of already answered question
              • etc.
              • 4. Re: Scoring...
                Richard Leeke

                For what it's worth - I don't in the least care if Shawn or anyone else for that matter overtakes me. The scoring system is a bit of fun - it's just intended to reflect the fact that the people providing the answers are donating their precious time - and are helping people. Helping someone who's stuck is just as useful to that person if the problem is easy or hard - they were still stuck. And who's to say that my time is worth more than Shawn's time?

                 

                I'd far rather the Tableau folk spent their time on continuing to make their excellent product even more excellent than worrying about this. As I'm sure they would.

                 

                But I appreciate the thought, Shawn.

                1 of 1 people found this helpful
                • 5. Re: Scoring...
                  Joe Mako

                  Shawn,

                   

                  In my opinion, the answer you provided in the example thread you linked to is worth the max points. You added value, you helped someone, with a quick response time, and it was a good answer. You volunteered your time, and it is worth wile, you earned those points, and I appriciate your contributions, you are helping to make this forum a better place.

                   

                  by the way, I think Stack Exchange has the best approach with reputation, here is an old post on their approach: http://www.codinghorror.com/blog/2011/10/the-gamification.html

                  1 of 1 people found this helpful
                  • 6. Re: Scoring...
                    Jonathan Drummey

                    Hi Shawn,

                     

                    I agree that the points system does not accurately reflect the skill/knowledge level of the person responding, and I'm not sure how well it can or how hard Tableau (and/or the forum posters) should try to get there. For example, there might be someone who has been using Tableau since 1.0 doing amazing stuff who hasn't been that interested in posting until now, and they'd likely have a more accurate response than someone like me who's been at it for a lot less time.

                     

                    I've been listening to Horace Dediu's podcasts at http://asymco.com and he's introduced me to the "Jobs to be Done" theory. For me, as a questioner the "job to be done" of the forum is to get my question sufficiently answered given my skill level, framing of the problem, stress level, ability to understand the offered solution(s), response time, etc. Ultimately, the number of points or bars that a responder has doesn't really matter as long as my need is met, and given the software we're working with it's usually easy enough to test out whether one or another solutions to my question works. The "reputation" that the points and bars are communicating might lead me to try out the solution from the person with more points first, but that's about the only difference.

                     

                    That's why I agree with Richard and Joe - in answering a question, any question, you are providing a service of value, and the points and bars reflect that.

                     

                    Jonathan

                    1 of 1 people found this helpful
                    • 7. Re: Scoring...
                      Shawn Wallwork

                      This got quite a bit more discussion than I ever thought it would, and much of what has been said has made me re-think the whole topic. I completely accept the argument that answering any question is a worthwhile and valuable endeavor (not to mention most times actually enjoyable). And if the scoring system were totally about 'participation' then I probably wouldn't have posted this topic to begin with. What I've come to realize is I'm personally uncomfortable with the labels associated with the points (and what they imply). I know they are also suppose to be just for fun, and currently being a 'Grand Vizier' is a vague enough label it does no harm. But a few months from now there's the danger of me attaining Joe's 'Data Guru' label, which would be complete hogwash! (not to mention very misleading to new-comers). So before things get out of hand, I'd like to suggest some tweaking is in order. How to tweak it, I'll leave that to others.

                       

                      A couple of other notes:

                       

                      Joe thanks so much for the kind words and encouragement, and the link to Stack Exchange. (I'll have to be careful not to get sucked into that bottomless fountain of knowledge!) I think you're right their scoring system would be an improvement over Tableau's. But I must say I much prefer the graphic simplicity of this forum; it has a Google-esque quality to it, and is easy to use/navigate.

                       

                      Richard and Jonathan, I agree this is nothing Tableau should spend much time on, at least not until they deliver the top 15 ideas that are currently pending .

                       

                      Finally, I did try to establish another account so I could start over at the bottom, but Tableau locked it up right away. Which, as Dimitri knows, is probably for the best (I would have been sorely tempted to nudge a few of my favorite Ideas up the vote ladder a bit ).

                       

                      All in all this has been an interesting discussion, I wonder what the Tableau web/forum folks think about this whole issue?

                       

                      --Shawn