HAhaberman·Mar 27, 2014
I find it illuminating to look at the exact values of the floating point numbers. This takes away an air of mystery: floating point numbers do in fact have very specific values, which we can inspect exactly.
Here's 0.1 + 0.2 + 0.3:
0.1 (0.1000000000000000055511151231257827021181583404541015625)
+ 0.2 (0.200000000000000011102230246251565404236316680908203125)
= 0.3000000000000000166533453693773481063544750213623046875
rounded: 0.3000000000000000444089209850062616169452667236328125
+ 0.3 (0.299999999999999988897769753748434595763683319091796875)
= 0.600000000000000033306690738754696212708950042724609375
rounded: 0.600000000000000088817841970012523233890533447265625
The operands can't be represented exactly as floating point so they are rounded to the nearest floating-point number before t
Read the rest
he addition even happens. And once the addition does happen, it is rounded to the nearest representable floating point value.
Now we can look at the same for 0.1 + (0.2 + 0.3):
0.2 (0.200000000000000011102230246251565404236316680908203125)
+ 0.3 (0.299999999999999988897769753748434595763683319091796875)
= 0.500000000000000000000000000000000000000000000000000000
rounded: 0.500000000000000000000000000000000000000000000000000000
+ 0.1 (0.1000000000000000055511151231257827021181583404541015625)
= 0.6000000000000000055511151231257827021181583404541015625
rounded: 0.59999999999999997779553950749686919152736663818359375
It's interesting to note that the first addition ends up being exact! This is a bit lucky, because 0.5 is exactly representable in floating point and the initial operands got rounded in opposite directions. It's also interesting that 0.6... only turns into 0.599999... at the point that we round the answer.
You can play around with this stuff conveniently using the Python "Decimal" module.
$ python
Python 2.7.5 (default, Aug 25 2013, 00:04:04)-
[GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.0.68)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from decimal import Decimal, getcontext
>>> getcontext().prec = 1000 # To prevent truncation/rounding
>>> Decimal(0.1)
Decimal('0.1000000000000000055511151231257827021181583404541015625')
>>> a = Decimal(0.1) + Decimal(0.2)
>>> a
Decimal('0.3000000000000000166533453693773481063544750213623046875')
>>> Decimal(float(a)) # get the value rounded to the nearest fp number
Decimal('0.3000000000000000444089209850062616169452667236328125')
You can also use my tool "dumpfp" (https://github.com/haberman/dumpfp) to inspect the value of a floating-point number in detail:
$ ./dumpfp 0.1
Single Precision (IEEE 32-bit):
raw = 0x3dcccccd
sign = 0x0
exponent = 0x7b (-4)
significand = 0x4ccccd
VALUE CALCULATION =
significand (1 + 5033165/2^23 (1.60000002384185791015625))
* 2^exponent (2^-4)
= VALUE (13421773/2^27 (0.100000001490116119384765625))
Double Precision (IEEE 64-bit):
raw = 0x3fb999999999999a
sign = 0x0
exponent = 0x3fb (-4)
significand = 0x999999999999a
VALUE CALCULATION =
significand (1 + 1351079888211149/2^51 (1.600000000000000088817841970012523233890533447265625))
* 2^exponent (2^-4)
= VALUE (3602879701896397/2^55 (0.1000000000000000055511151231257827021181583404541015625))