The Nature of Measurement: Part 6: Level of Certainty

This article will clarify the concept of level of certainty (also called confidence level or simply probability) regarding measurement errors. Understanding this concept is necessary for a surveyor to make statements that are correct in a theoretical/scientific sense with regard to measurement error on surveys, and for interpreting error statements regarding instruments or specific surveys. As with other concepts related to errors, this one must be thoroughly understood before positional tolerances and errors can be analyzed.

Before we attempt to apply this concept to the world of measurement, it will be useful to examine it in a general sense. To understand how it affects our everyday activities and choices may make it easier to come to terms with this somewhat abstract concept as it applies to measurements. As an example, we have learned to process a statement by the "weather man" as to the percent chance of rain, and this helps to decide whether or not to carry an umbrella. The cost of automobile insurance is affected by statistics on traffic accidents. The probability of a professional baseball player making a home run at any randomly selected time at bat differs from a probability based on data collected from past performances. There are many examples to illustrate how statistical probabilities are calculated and applied every day.

Sometimes we are part of the statistics that determine probabilities. At other times, we function within an environment where the probabilities affecting us are determined by forces beyond our control. Most of the time, however, we can make choices of activities or other things that affect our lives, thus altering the probabilities for success or failure. Each environment and each choice has a different set of probable outcomes, such as satisfaction, health, etc.

Probability Explained Philosophically

To further put probability into perspective, let us examine "certainty" itself. I have found that secure, well balanced, mature people generally have some doubt on most decisions. They realize that most are made with incomplete or distorted information, and through personal filters of bias, fear, pride, prejudice and individual experience. Such a person might feel confident about well researched and considered judgments, but knows that some of the data and/or human judgments are nevertheless flawed.

Stated a little differently, a person who is 100 percent certain is a fool. Such a person is closed minded, allowing no room for change based on new knowledge, evidence, insights, facts, or debate. It is healthy to have self-confidence when judgments are well considered, but some measure of caution and doubt is healthy too. I submit that 100 percent certainty is not possible in the real world, where human judgment and perception, plus data manipulation, are usually involved (the exception being that which we choose to believe out of an act of faith). If this is true regarding human decisions, it surely must also be the case regarding measurement science. When we see it from both a philosophical and a measurement science approach, reconciling the two as one reality, it makes sense.

Examples of Probability

Some like to quantify everything; others like to use "gut feelings" or experience to decide between alternatives. Either way, rational people make decisions, consciously or unconsciously, subjectively or scientifically, applying some form of "level of certainty."

As an example of setting probabilities using feelings or past experience only, without any quantification or controlled experiments, suppose I am ordering dinner at a restaurant. Past experience affects my menu choices. I don't think well of the corned beef, but my mouth waters when I think of filet mignon. To put some numbers on this, I may be subjectively and quite unconsciously assigning a 5 percent probability that I will like the corned beef and cabbage (with 95 percent probability that I will not), and 90 percent probability that I will like the filet mignon. Note that I did not go all the way with a 0 or 100 percent on either, since experience has taught me that maybe some cook just might be able to prepare corned beef and cabbage so that I can enjoy it—unlikely, but worth a 5 percent chance. Likewise, I know that some chefs cannot get "medium rare" right, or that the particular cut of beef will not be choice, so I might assume that one time out of 10, I will be disappointed with the steak.

As a simple example of something having clear numerical or quantifiable probability associated with it, suppose I put one white and one black marble in a box and reach into the box and select one. What is the probability of getting the white marble? Of course, it is 50 percent. If I play "Russian roulette" with a six-shooter and one bullet, what are my chances of surviving the game? Assuming the bullet and gun are not defective and my head is not too hard, it is 5/6 or about 83 percent. What is the probability of drawing an "ace" from a full deck of cards? The answer is 1/13. Gamblers are constantly confronted with probability. Every reasonably intelligent person should know (intellectually, but perhaps not always emotionally), that we cannot win in the long run at the gambling tables or machines, since the probabilities are created in favor of the house. Gamblers who deceive themselves that they can win in the long run do not understand (or choose to ignore) the concept of probability and the nature of the business.

This last statement, perhaps reveals why it may sometimes be difficult to teach level of certainty to some people. If a person denies that this variable exists and must be reckoned with, that person is difficult to teach much about surveying measurement, or anything else where risk is involved. If a person understands that all actions, choices and random occurrences have that dimension associated with them, and further understand that some numerical (and often easily quantifiable) estimate of it can be made, applying the concept to measurements comes relatively easy. Then maybe it will help to overcome the temptation to approach the casinos or any game or activity where the odds have already been stacked against you! How do you think they got the money to build those casinos in the first place?

Level of Certainty in Measurement

Any measurement has three values or numbers associated with it. That's right—not one, not two, but three. First is the estimate (all measurements are estimates) of the size of the quantity. Then, since a measurement is an estimate of a continuous (another statistical term) number, there must be some estimate of the range of error. This range of error is often called the "uncertainty," but more commonly simply the "error."

Next is the third dimension to a measurement—level of certainty. We see it in National Map Accuracy Standards, where it is stated that "90 percent of the well defined horizontal positions shown on the map shall be within 1/50 inch," and "90 percent of the elevations interpolated from the map shall be within 1/2 the contour interval." This is a statement saying we are 90 percent sure of our accuracy, within stated limits. Only nine out of 10 of the points tested would need to pass the test. One could fail and we would meet the standard. GPS standards are on the 95-percent confidence level, as regards the relative positional error between adjacent points. The 95-percent confidence level seems to be emerging as the favored level. Some instrument accuracies are cited by manufacturers at the 68-percent confidence level. You must know the level of confidence attached to any cited errors to properly interpret the accuracy or precision statement.

Standard Deviation and Probability

Probabilities are fairly easily computed for simple situations, such as heads-versus-tails in a coin toss, but other situations are much more unclear and complex. Statisticians, researchers and surveyors use controlled experiments to determine repeatability and probability of certain outcomes or occurrence of certain selected events. Once a sample of something is taken, the standard deviation of the readings can be computed. In measurement, the standard deviation is a measure of the precision of the method, or the way random errors behave in the procedure. The larger the standard deviation, the larger are the random errors, and thus the lower the precision. If the normal probability curve is plotted, statististicians tell us that the area under the curve between plus and minus "sigma" (standard deviation) is 68.3 percent of the total area. This is not just some esoteric theory. It works. In making tests of the reading precision of 1~ theodolites, I usually take 25 readings, estimating to 0.1~. After calculating the standard deviation, a count of the readings falling between plus and minus sigma (with respect to the mean) is nearly always exactly 17 (which is 68 percent of 25). I have never found it to be less than 16 nor more than 18. You can be off a little since the sample set is, after all, finite in size, not infinite!

Another way to understand the theory is to think of writing each of the 25 readings on a small piece of paper, and placing them in a box. What is the probability that one drawn at random will be between plus and minus sigma, with respect to the mean? The answer is 68 percent. The 68 percent probability here is as dependable as the 50 percent chance of drawing the white marble instead of the black marble, or a 25 percent chance of drawing a card in the suit of clubs rather than hearts, spades, or diamonds.

Varying the Level of Certainty

Most of us would prefer not to work with only 25, 50 or even 68 percent certainty. Maybe you are Dwight D. Eisenhower and someone asked you how you felt about your decision to proceed with the Normandy invasion and you said only 50 percent. Not too confident. In reality, he probably had his doubts, but was more certain that 50 percent or 68 percent. Maybe even 90 percent or 95 percent? Was the man 100 percent sure? I doubt it. Ike was no fool.

In measurement, the way we determine certainty is much easier than when making difficult decisions in life (invading Europe, getting married, etc.). Aside from statistics, how could you be more confident of an error statement? The answer is to increase the size of the error estimate. Isn't it logical, for example, that I can be more sure of my ability to pace a 100-foot distance within 10 feet than within 1 foot? It isn't logical that the level of certainty would be the same for these two different error estimates. What would be the difference in certainty? Some people might just say "I'm sure of being within 10 feet, but wouldn't bet my life on being within one foot." That doesn't sound very learned for a supposed expert in measurement, and rather quaint as a statement of error certainty on a survey plat! A "gut feeling" might be used, such as 99.9 percent sure of the 10-foot accuracy, but only 50 percent sure of the one-foot accuracy. But gut feeling isn't good enough for professionals in the practice of measurement science, and not very defensible under the scrutiny of the prosecuting attorney who hired one of those college graduates who had three courses in this stuff.

Statistically Analyzed Probabilities

The means to arrive at any level of certainty (short of 100 percent) lies in statistical analysis. The theodolite reading error test can be used to illustrate. I have done many such tests and my standard deviation always comes out very close to 0.8". As stated, the standard deviation is at the 68.3 percent confidence level. I can declare to the world that my precision in aligning the theodolite micrometer marks and estimating one reading is ±0.8". But I am only 68.3 percent confident of this statement. So, how do I gain more confidence? By increasing the error estimate! Statistics shows that, to be 90 percent sure, I multiply the 0.8" by 1.645. To be 95 percent sure, I double it. To be 99 percent sure, I use 2.5 sigma. The "3-sigma" error is about 99.7 percent certainty. It is impossible to have 100 percent certainty, unless the error range is plus or minus infinity! Isn't that a wonderful insight? Remember, we said that anyone who is 100 percent sure is, by definition, a fool. Finite people cannot reach out to infinity. This fact, when fully digested, not only gives great insight to the realities of measurement, but also to the realities of life—it keeps you humble and open-minded.

Using these for different levels of certainty, I can declare that I can observe one theodolite circle reading to ±1.3" and be 90 percent confident of this estimate, ±1.6" and be 95 percent confident, ±2.0" with 99 percent confidence, ±2.4" with 99.7 percent confidence, etc. Notice that to be highly confident, such as 99.7 percent sure, requires that the error range be nearly doubled from what it is at the 90 percent level.

The standard deviation is a powerful analysis tool. It gives a good estimate of the predicted repeatability of a measuring method or random error from specific error sources. Using the standard deviation, one can then calculate expected errors for other levels of certainty. Statistical tests to isolate random error sources and quantify them is important for controlling random errors, making estimates of positional errors, deriving measurement specifications, applying weighted least squares adjustments and describing the quality of measurements. The details of calculating standard deviation will be the subject of a future article in this series. We should add now, however, that the same tests can be made for many measuring operations, or for investigating individual errors involved in such operations. Furthermore, we aren't finished when we analyze just one error source (such as reading error). The several applicable errors in any measuring operation must be individually investigated, then combined through a process called error propagation.

A Word About Accuracy

Any statistically derived error is an estimate of the precision of the measurement when investigating only the random errors, and an estimate of the accuracy when evaluating the extent that systematic errors have been discovered, quantified and removed from the observations. In actuality, this estimate is some function of both precision and accuracy.

The 90 percent error in one reading of a theodolite, even when combined through error propagation with pointing, bubble centering, and other error effects, is still just a precision estimate. If the optical plummet of the instrument was out of adjustment or the targets were not centering properly, the angles would have corresponding inaccuracies. My confidence level regarding the reading errors relates only to repeatability or precision of that one variable. If I want to make a statement of accuracy, I need to investigate and quantify errors that behave systematically, using controlled tests, the result being estimates of the random errors in the systematic errors.

Summary and Conclusion

I hope this article will serve to direct the thinking of surveyors and their technicians concerning the "third dimension" of surveying measurement. To fully understand errors and keep them in control, make defensible and theoretically correct statements about the errors, and apply concepts such as positional error, we must always recognize the three numbers associated with any measurement. The nature of measurement is one of dealing with errors and probabilities. All measurements are but estimates or professional opinions, with the uncertainties that accompany any opinion. The extent of uncertainty should be identified in any opinion or perception. It is insufficient to merely cite a measurement without an estimate of its error. And, it is just as bad to estimate the error without also attaching a confidence level to it.

We live and operate daily in a world of probabilities. We constantly make (or should make) judgments based on some subjective or quantifiable percent probability of the outcome of the decisions. Our measurements are no exception. When we learn to live with, think by, and apply probability, we will be more successful in our daily decisions, be more honest with ourselves and others, and maintain an accurate image as professional surveyors.

» Back to our Nov/Dec 1997 Issue