4 Replies Latest reply on Oct 11, 2013 1:36 AM by Peter Juel Jensen

    Do Tableau have trouble adding numbers?

    Peter Juel Jensen

      I see a strange miscalculation when I create extract on data bigger than approximately 100.000 rows.To illustrate my point I made a random dataset in excel with lots of random numbers - but all between 0 and 1 and with 2 decimals (attached xlsx file).

       

      When I connect live to the dataset Tableau is showing the following sum (attachment 1).

      Live connect.png

      It is strange that a sum of 2 decimal numbers ends up as a 11 decimal number. When I do the extract the number change, but it is still wrong (attachment 2)

      Extract.png

       

      On larger datasets the difference between live connections and extracts is larger (25M rows with a total sum of 114B calculates 0,25 wrong) and I see the same error when I use the SQL server as the live connection. I used the desktop version 8.0.4 for this example.

       

      Are there a reason for this? And can I avoid it?