7 Replies Latest reply on Apr 12, 2016 2:04 AM by ANTONIO GAZQUEZ

    Small samples: how to identify them?


      Hi all!


      Although its my first time writing here, I'm quite familiar with the community since I usually need to seek advice. First of all I'd like to thank for so many help! One of the nice things about Tableau is the community!


      But let's get to the point: during the last week I've been struggling with how to notify/highlight in the dashboards small samples. As you probably know, showing means or percentages with small samples can lead to dangerous conclusions. For me and my clients, it's quite important to know when a figure is supported by a robust sample or not. I'm quite sure some of you have faced this problem.


      Normally I have weighted means and percentages so sometimes knowing samples it's not that easy. I'd like to have a variable which is sample<30 o sample>30. The thing is that I need this variable working for all measures. Then, I'd like to use this variable with colour or filter just to notice when I have small sample or not.


      Since I'm not quite sure about how clear is the explanation, I've attached a workbook so you can have a look at data structure.


      Thank you very much!

        • 1. Re: Small samples: how to identify them?
          Pablo Saenz de Tejada

          Hi Antonio,


          I guess you have 1 row per respondent and per question right? so you are filtering each sheet by the concrete question you want to analyze / visualize. Normally when I want to see sample sizes, what I do is a count of number of records for the concrete question I want. COUNT([Numer of records]) or if you have a Respondent ID, a count distinct of respondent ID (should be the same if you have only 1 row per respondent). If you include that calculation in the tooltip, it will show you the size of the sample for each mark in the visualization.


          Another possibility is also to include a calculation to use it as color for example (or also as an additional label). So for instance you can create a calculation like


               COUNT([Number of Records]) <= 30


          This will give you a True if the number of records (so, the sample if you have only 1 respondent per row. Again you could use a COUNTD of respondent ID if you have a respondent ID field with a unique identifies per respondent) is less  or equal to 30, and a false if it's greater than 30.  Then you can use this in color and you will color differently the small samples and big samples.


          Additionally, you could create a field like IF COUNT([Number of Records]) <=30 then "Small Sample" ELSE NULL END and use that field in labels in addition to the measure. So it would show in the label the text "Small Sample" in those cases.


          I hope you find this useful. Let me know if you need additional information.




          1 of 1 people found this helpful
          • 2. Re: Small samples: how to identify them?

            Hi Pablo!


            Thank you for your help! This approach will almost solve my problem. However, I have some other worksheets where I'm not able to show the count. I'll explain:


            This situation takes place when I want to visualize in a table different measures, like this one, for example:




            As you can see, I have 4 different means from 4 different variables. If I use here count of number of records the result will not show the real count of the variable (count here will be the same for every single variable).



            If I calculate the real count of every variable in another table I can see how are different.


            Is there any way to visualize in the first table the count for every variable? I can't come up with any idea...


            Thank you!

            • 3. Re: Small samples: how to identify them?
              Jennifer Pattershall-Geide

              Hi Antonio - I have a couple of techniques for displaying Ns in my survey data. One is to have an "N counter" on the dashboard that's tied to a filter action on my graph, so when someone clicks on a question they get exact number of people who answered that question. For instance, in the two images below, you'll see that the N changes when a different question is selected. Note that the N counter is just a separate worksheet displaying the number of records.

              N Count 1.PNG

              N Count 2.PNG


              Another possibility is to put the N into the chart/table, such as below:

              N Count 3.PNG

              To use this approach just make sure your count is discrete rather than continuous. Depending on the nature of your data you may need to use an LOD expression (which I have to do to get a total across all answers on Likert scale, rather than counts for each answer separately). The specific equation (to which I credit Steve Wexler) is:


              {EXCLUDE [Answer] : SUM([Number of Records)] }


              or, if you have pivoted data (as I do):


              {EXCLUDE [Answer] : COUNTD([ID] }


              I'm sorry I can't share a workbook due to the nature of my data, but if you'd like more details on either of these approaches please let me know.

              1 of 1 people found this helpful
              • 4. Re: Small samples: how to identify them?
                Pablo Saenz de Tejada

                Hi Antonio,


                It's weird because if I calculate the count of each one I get the same number (see image, and even if I delete the filter of "Agencia"):



                I have checked the data base and it seems to have the same records for all the different measures?




                • 5. Re: Small samples: how to identify them?
                  ANTONIO GAZQUEZ

                  I'm sorry Pablo. I had the same problem. In the original database in Excel, when I have null values I have this: "." It looks like Tableau is not reading this as null value. Normally I don't have my data in Excel, I just did it this time for the example so I didn't notice.


                  Anyway, please find attached a new workbook, this one is correct. There you'll be able to check what I meant in my previous post.


                  Thank you again!

                  • 6. Re: Small samples: how to identify them?
                    Pablo Saenz de Tejada

                    Hi Antonio,


                    sorry, had some busy days. I'll have a look at it as soon as possible.

                    • 7. Re: Small samples: how to identify them?
                      ANTONIO GAZQUEZ

                      Thank you very much Pablo! Anyway I managed how to solve it with some workarounds. For now, I have something like this:




                      I have two tables:


                      • One on the left side, with one weighted measure per row.
                      • One on the right side, with the count of every measure.


                      So if you click on the number you will get the table filtered by date on the right.


                      If you have a better solution please let me know! I'd like to have this information in tooltip directly but it seems to be impossible.


                      Thank you!