Question

Kyle divided 2 3 on his calculator and got .6667. Where did the 7 come from?

a. Kyle made an error.
b. The answer is exactly .6667.
c. The calculator rounded the answer.
d. The calculator truncated the answer.

Answer 1

The calculator rounded the answer.

The exact value of
`(2)/(3)` is
`0.\overline{6}`, i.e. a zero followed by infinitely many 6s.

The calculator can store only 4 decimal digits, so it has to approximate the number. By doing so, it has to "choose" between 0.6666 and 0.6667.

Since the next digit is a 6, the approximation is 0.6667.

In fact, if you have to approximate with four decimal digits, you have to look at the fifth. If it is between 0 and 4, you use your fourth digit. For example, 0.35742 is approximated by 0.3574.

If instead the fifth digit is between 5 and 9, you raise the fourth digit by one: 0.57836 is approximated by 0.5784