2007-10-16

0.01 * 10 != 0.1

0.01 * 10 == 0.099999994

...if you are using single precision floating-point variables, also known as floats in C#. The solution? Using the decimal data type (or, if you are a Java developer, the BigDecimal class).
Although I didn't test it, this seems to be an issue in several similar programming languages, which use IEEE 754 data types. The problem is that not all real decimal numbers are directly representable in binary (e.g. 0.01), and as such are approximated. The solution involves using a special binary notation to support the accuracy required for financial applications.

Until next time, may you enjoy being surprised with something you though for sure it was true, but it was not, and you should have known better.

2 comments:

  1. E quanto dá 4 + 2 - 1 + 6 * 2?

    ReplyDelete
  2. Muito interessante!!!

    ReplyDelete