0 Replies Latest reply on Jan 11, 2012 2:40 PM by Brian Ramagli

    It'd be nice for more than 15 points of numerical precision

    Brian Ramagli

      Per http://onlinehelp.tableausoftware.com/v6.1/pro/online/en-us/help.htm there are only 15 digits of precision.

       

      This is very noticable if you have unsigned big integers representing what should be very distinct values (the last N digits are replaced with 0). This makes it nearly impossible to take data extrapolated from Tableau, and cross reference it back to the database, or use these numbers in other meaningful ways.

      I suspect this is also an issue for doubles of high precision.

       

      This has been posted as a question with no response here. Luckily some digging and time with tech support has narrowed down it being a "bug" even if documentation wants to pass it off as a warning.