This is likely due to rounding. The number actually has many more decimal places than just the 6 shown. If you were to show all the decimal places (16) for each field, the result Tableau is returning will be correct.
Hope this helps!
1 of 1 people found this helpful
Confirmed - if you format the display to show more decimal places and use these it shows Tableau is correct
This could be very misleading by changing the format but the calculation is still using 16 decimal places. I can't imagine anyone showing all 16 decimal places. It would be nice to have a feature that would calculate off of the format.
Taking Less than $20,000 as an example (6 decimal places): .055223 / .021127 = 2.613859 (Tableau results show 2.613914)
Instead of the 16 decimal places:
0.0552233168612886 / 0.0211266791652893 = 2.6139137357668296259301221886671
I've been in too many presentation where an executive would take a calculator and compute the number on the dashboard. Lose all credibility when the numbers don't match.
You would never ever want a feature like that.
- You should be rounding anyway - >5 significant digits is too many
- You should be equipped with the answer that this is a rounding issue
- If more than 5 sig digits are important, you would lose credibility if you used rounded numbers...