Texas Instruments calculator verses Java double data type

There seems to be no readily available information on this topic. Why are calculators more accurate then the double data type? For example, in Java if you take (1/999999)*(999999) you get .9999999999999999 when the real answer is 1. This doesn’t seem to be a problem with a Texas Instrument calculator though. I can’t imagine that the memory is fundamentally different. And there isn’t enough memory in the world to store an infinite decimal number. Is there some sort of software workaround that lets the calculator assume infinite values? Is it possible to replicate this process in Java?

1 Like

Simple, they don’t use doubles.

Calculators in particular don’t store the numbers in binary at all because of precision issues: There’s numbers which can be easily expressed in decimal but take far more bits in binary. Storing numbers in decimals is common for a lot of software where rounding errors would be problematic; just think of e-banking.

So there’s really not much to it. Conceptually you could store the numbers as arrays of 8 bit values, each of which represents a single digit. Then perform all the calculations in software. More efficient ways exist of course.


I don’t have a real comment, just thanks for the interesting read.

That is very interesting. I would like to create a custom calculator application. With a GNU license of course.

1 Like

There’s finished libraries for dealing with large numbers. A few resources:

Also note that doubles are not actually the widest floating-point numbers available. x86 natively supports 80 bit floats (exposed via “long double” in C). This isn’t really relevant to your calculator, but don’t let java narrow your thinking :wink:

1 Like

yeah floating point rounding errors are always a bitch.

in C you could do long long double.

1 Like

That’s what the Like button is for.

I too have no comment.

1 Like