5 Replies Latest reply on Apr 10, 2013 12:49 PM by Clare Bayley

    Why is Tableau miscalculating basic numbers?

    Clare Bayley

      I'm seeing some bizarre behavior and am at my wits end trying to figure it out.

       

      I'm working with a table of sales data (sorry, can't post my workbook) and one of my columns is an integer transaction amount (so, $ * 100).

      The numbers appear correctly when I view the raw data both in Tableau and through SQL, but as soon as I perform any operation on the number, the result is incorrect but consistent. If I create a calculated field of [amount]+0 or [amount]*1 or even ABS([amount]) suddenly the results are as follows:

       

      0 -> 17,869

      1,795 - > 17,960

      3,499 -> 18,469

      19,478 -> 3,990

      and perhaps most strangely of all,

      17,955 -> Null

       

      The same thing happens if I force the data type to string - the resulting string is the weird incorrect number. It's almost as if it's trying to read them as code for a number instead of the number itself but I can't detect any pattern. I've tried starting fresh with a new data source and got the same problem.

       

      What. :\

       

      Attached is a screenshot of the behavior -

      rebill_amount is the raw data, and shows the correct number.

      rebill_amount_fixed is the attempt to calculate it - ABS([rebill_amount]) in this case, although *1 or +0 gets the same result.

      TF_RebillAmount is [rebill_amount_fixed]/100