Skip to the content of the web site.

1.1 Precision and Accuracy

Introduction Theory HOWTO Examples Questions Matlab Maple


Two words we will be using to describe how good a measurement or approximation is to an actual value are precision and accuracy.




  • Chapra, Section 3.2, Accuracy and Precision, p.53.

Note: the definitions of precision and accuracy on MathWorld use Mathematica's definitions which do not correspond to the standard definitions in the literature.


Whether a number is an approximation of an actual value as a result of a calculation or a measurement, we must be able to describe the approximation. We will use the term precision to describe how close an approximation could be to an actual value and accuracy to describe how close it actually is.


Given a series of measurements or approximations, there is an error associated with each value. We will use the term precision to measure the spread of errors relative to each other. Precision is independent of the actual correctness of the measurements or approximations.

Precision is usually described in terms of the number of digits used to make a measurement or approximation though it can also be described in terms of the standard deviation of the errors. For example, an amp-meter with a three-digit display is more precise than an amp-meter with a two-digit display. With a 30 cm ruler, you can usually measure the distance of an object down to the closest millimetre, and perhaps even down to a quarter of a millimetre. Such a means of measuring is not as precise as if you were using a micrometer to measure the same distance. Additionally, a poorly calibrated micrometer is still more precise than a 30 cm ruler.

In Matlab, if you type format long and attempt any interesting mathematical computation (something which does not result in an integer) you will note that the number of digits stored is always the same: that is, the precision is unchanged from question to question.


Accuracy describes how close an approximation is to an actual value. For example, the speedometer of a Golf is calibrated with ticks every 5 km. Thus, I could estimate that my speed is 59 km/h, or, at the same time, I could estimate that my speed is 58.888483 km/h. As the actual speed may be 58.959383⋅⋅⋅ km/h, we note that the error between both approximations and the actual speed are 0.040617 and -0.0709 and therefore both answers have approximately the same accuracy. It may be that the actual speed is closer to one of the two answers, but this would be coincidence.

In Matlab, because the precision is fixed, the goal of any algorithm is to find an accurate answer.

Implied Precision

When writing down a measurement as a decimal number, there is an implied level of precision, namely, 0.5 units in the last position. For example, a measurement of 23.534 implies that the maximum error is correct to at least 0.0005. Alternatively, it may be convenient to write down a measurement with the maximum error explicitly given: 23.534 ± 0.012, implying that the actual answer lies in the interval (23.522, 23.546). While such a notation is useful for the actual study of error propagation, this will not be used much in this course.

Implied precision is a measure of absolute error, covered in the next topic.

In Matlab, the implied precision is always the same. The algorithm used must give an approximation as to the actual accuracy.

Precision and Accuracy

Precision and accuracy are two independent quantities: one can be precise but inaccurate or vice versa. For example, suppose five readings of a voltage across a resistor yields the values 7.3, 7.4, 7.3, 7.3, 7.2. If (due to a burnt-out LED) these were accidentally recorded as 1.3, 1.4, 1.3, 1.3, 1.2, then the recorded measurements would still be as precise as the actual measurements, however, the accuracy is significantly reduced. As another example, the radar guns used by police may measure the speed of five vehicles traveling at 100 km/h as 96 km/h, 99 km/h, 105 km/h, 102 km/h, and 103 km/h, while the new laser speed detectors measure the same speeds as 100 km/h, 100 km/h, 99 km/h, 100 km/h, and 101 km/h. The first set of readings are accurate but not very precise while the second set of readings are accurate and more precise.

When displaying an approximation, it is recommended that the precision reflects the accuracy. For example, you may use a computer to estimate π using a series expansion. For example, summing the reciprocals of the squares of the first hundred integers is 9.8099034011 and is an approximation to π2/6. Thus, 3.1320765318 could be used as an approximation of π however, our approximation is not very accurate, and it would be better to state that the approximation is 3.1 .



Precision describes how many digits we use to approximate a particular value. It is very possible to have a very precise approximation which is not very accurate.


Accuracy describes how close an approximation is to a correct answer. In the next section, we will see how we can describe accuracy using either absolute or relative error.


1. Which number has more precision and accuracy as an estimator of π, 3.1417 or 3.1392838?

The second number has higher precision, but it would appear that the first more accurate.

2. Using Matlab and the formula you learned from Calculus to approximate the derivative, which of the following are more precise and which are more accurate in approximating the derivative of sin(x) at x = 1?

>> format long
>> (sin(1+1) - sin(1))/1      
ans = 0.0678264420177852
>> (sin(1+1e-5) - sin(1))/1e-5  
ans = 0.540298098505865
>> (sin(1+1e-10) - sin(1))/1e-10
ans = 0.540302247387103
>> (sin(1+1e-15) - sin(1))/1e-15
ans = 0.555111512312578

From Calculus, you learned that the derivative of sin(x) is cos(x), and therefore the correct answer is cos(1) = 0.540302305868140, but looking at the accuracy, we see that the third answer is most accurate as an approximation of cos(1) and the first and last are the least accurate. All four answers have the same precision, 15 decimal digits,

3. The distance from Ottawa to Waterloo, following 400-series highways, is approximately 332 mi. As 1 mi = 1.609344 km (exactly), it follows that the distance is approximately 534.302208 km. Discuss this conversion with respect to precision and accuracy.

Both answers have the same accuracy, even if the second approximation has much more precision. Thus, the extra precision is unwarranted and 534 km would be a sufficiently good approximation.

4. To demonstrate the relationship between accuracy and precision, suppose we have three measuring devices: a standard 30 cm metal ruler, a old and warped 30 cm wooden ruler, and a micrometer. If we were to use these devices to measure 100 objects of which we knew the exact length and were to plot the errors of the approximations of our measurements, the errors may be distributed as shown in Figures 1, 2, and 3.

Figure 1. Plot of errors in measurement using a 30 cm steel ruler.

Figure 2. Plot of errors in measurement using an old and warped 30 cm wooden ruler.

Figure 3. Plot of errors in measurement using a micrometer.

Note that both 30 cm rulers have approximately the same spread of error, and therefore may be said to have the same precision, however the errors using the wooden ruler are biased, and therefore we may say that the wooden ruler is less accurate. If we compare the steel ruler and the micrometer, we note that for each, the errors are centered around zero, however, the spread of the error with the micrometer is smaller by a factor of 100, and therefore, we may say that the micrometer is more precise than the steel ruler.

(1 mm = 1000 μm)


1. Which number has more precision and which has more accuracy as an approximation of e, 2.7182820135423 or 2.718281828? (first more precise, second more accurate)

2. Your partner uses a ruler to measure the length of a pencil and states that the length is 20.35232403 cm. What is your response to the given precision?

3. Suppose you are measuring the current flowing through a resistor with one amp-meter which displays three digits but is 10 years old and was found in a damp corner of a store room, and another amp-meter which displays only two digits but was purchased from Sayals last week. Describe the expected precision and accuracy of the two readings.


Matlab uses a fixed amount of precision equal to approximately 15 decimal digits.

>> format long
>> 1.5/1.3    
ans = 1.15384615384615
>> sin(1)
ans = 0.841470984807897


Maple lets you determine how much precision you will use in a particular calculation by assigning to the Digits variable:

> 1.5 / 1.3;
> sin( 1.0  );
> Digits := 20;
                                 Digits := 20
> 1.5 / 1.3;
> sin( 1.0  );
> Digits := 100;
                                 Digits := 100
> 1.5 / 1.3;
> sin( 1.0 );