from Hacker News

Why does Excel get 213,458,046,676,875 x 29 wrong?

by christudor on 4/7/22, 3:08 PM with 2 comments

When I put =PRODUCT(213458046676875,29) into Excel, it gives me the answer 6,190,283,353,629,370.

The actual answer is 6,190,283,353,629,375.

What has happened?

  • by lavezza on 4/7/22, 3:47 PM

    Excel only stores 15 significant digits. Once you get to the 16th digit you'll just get a zero.

    Example: 6,190,283,353,629,370 + 1 will still be 6,190,283,353,629,370. 6,190,283,353,629,370 + 10 will be 6,190,283,353,629,380

  • by db48x on 4/7/22, 3:45 PM

    Floating–point numbers happened. Excel uses IEEE–754 floating–point arithmetic because it is fast and ubiquitous. One of the disadvantages of doing math this way is that as you get farther away from zero, the precision of the numbers goes down. It’s generally fine because most people don’t do arithmetic on 6 quadrillion and change, they do arithmetic on smaller, more approachable numbers.

    Look for a way to solve your problem using software that uses unlimited–precision arithmetic instead.