Sitemap

Floating Point Arithmetic Precision Issue

4 min readSep 28, 2024

--

Press enter or click to view image in full size
let x = 50, i = 0;
while(i<10){
i++;
x+=0.01;
}
console.log(x)
//Expected=> 50.1
//Actual=> 50.09999999999998

Step by Step explanation of above problem:

How Computers Store Numbers:

Imagine you have a special notebook where you can only write using 1s and 0s. This is how computers store numbers — in binary (base-2) instead of our usual decimal (base-10) system.

The Problem with Some Decimal Numbers:

Some numbers that are easy for us to write and understand, like 0.1 or 0.01, are actually very hard to write exactly using only 1s and 0s. It’s like trying to divide a pizza into exactly 3 equal parts — you can get very close, but there’s always a tiny bit left over or missing.

Press enter or click to view image in full size

In JavaScript (and many other languages that follow the IEEE 754 standard for floating-point numbers), the decimal number 0.1 cannot be represented exactly in binary. Let's explore why and how 0.1 is approximated in binary.

Steps for Binary Representation of 0.1:

  1. Understanding Decimal to Binary Conversion:
  • The integer part of a decimal is straightforward to convert to binary, but the fractional part (like 0.1) is more complex.
  • To convert the fractional part to binary, repeatedly multiply it by 2. The integer part of the result becomes the next binary digit (either 0 or 1), and the process is repeated on the fractional remainder.

2. Converting 0.1 to Binary:

  • Start with 0.1 (in decimal).
  • Multiply 0.1 by 2:
0.1 * 2 = 0.2 -> integer part: 0
  • So the first binary digit after the point is 0.
  • Now take the fractional part 0.2 and multiply by 2 again:
0.2 * 2 = 0.4 -> integer part: 0
  • The second binary digit is 0.
  • Repeat the process:
0.4 * 2 = 0.8 -> integer part: 0
0.8 * 2 = 1.6 -> integer part: 1
0.6 * 2 = 1.2 -> integer part: 1
0.2 * 2 = 0.4 -> integer part: 0
0.4 * 2 = 0.8 -> integer part: 0
...

This process keeps repeating, producing a repeating binary fraction.

0.0001100110011001100110011001100110011001100110011... (repeating)  

Why Does This Happen?

  • Binary System Limitation: Just as 1/3 cannot be exactly represented as a finite decimal number (it becomes 0.3333... repeating), 0.1 cannot be exactly represented as a finite binary number. The approximation keeps repeating infinitely.
  • Precision: JavaScript uses 64-bit floating-point numbers, which means it cannot store the infinite repeating binary expansion of 0.1. Instead, it truncates or rounds the value after a certain number of bits, introducing a tiny error.

Summary:

  • The binary representation of 0.1 is a repeating fraction: 0.00011001100110011....
  • JavaScript approximates this with 64-bit precision, introducing small rounding errors.
  • This is a consequence of the limitations of the IEEE 754 floating-point format used to represent numbers in JavaScript.

Computer’s Best Approximation:

So, when we tell the computer to use 0.01, it actually uses a number that’s very, very close to 0.01, but not exactly 0.01. It’s the closest the computer can get using its 1s and 0s system.

Adding Up Small Errors:

In our example, we’re adding this “almost 0.01” to 50, ten times. Each time we do this, we’re also adding a tiny error.

Accumulation of Errors:
These tiny errors, although very small, start to add up. It’s like if you were slightly off each time you measured an ingredient while baking — by the end of the recipe, the mistake would be noticeable.

The Final Result:

After adding this “almost 0.01” ten times, all these tiny errors have added up. Instead of getting exactly 50.1, we end up with 50.09999999999998.

Why This Matters:

For most everyday calculations, this tiny difference doesn’t matter. But in fields like finance, scientific calculations, or when dealing with large amounts of data, these small errors can add up to significant differences.

  1. An Analogy: Think of it like walking. If you try to walk exactly 1 meter ten times, you might end up at 9.99999 meters or 10.00001 meters. You’re very close to 10 meters, but not exactly there. Computers face a similar challenge with certain numbers.
  2. The Broader Implication: This is why you might sometimes see strange results when doing calculations with decimal numbers on a computer. It’s not that the computer is bad at math; it’s just that it’s forced to use a number system that can’t represent some decimal numbers exactly.
  3. In Practice: For most everyday uses, these tiny differences don’t matter. But when precision is crucial, programmers use special techniques or libraries to handle these calculations more accurately.

This phenomenon is a fundamental aspect of how computers work with numbers, and understanding it helps explain many of the quirks we see in computer calculations involving decimal numbers.

--

--

Aditya Yadav
Aditya Yadav

Written by Aditya Yadav

Software Engineer who talks about tech concepts in web development https://www.linkedin.com/in/aditya-yadav-01/