Much has already been written here about the process of capturing the change agenda and developing strategy maps. These important tools are valuable for communicating strategy across the organization. But they also serve as the foundation for identifying the performance measures that will motivate the behavior changes needed for strategy execution. And without a healthy skepticism, measures can mislead as much as they inform. Many remember that Mark Twain wrote,
“Figures often beguile me, particularly when I have the arranging of them myself; in which case the remark attributed to Disraeli would often apply with justice and force: ‘There are three kinds of lies: lies, damned lies, and statistics.‘”
While there is certainly deliberate distortion of statistics every day in the political sphere, we hope that the measures that become part of a strategy execution program are effective in both motivating behavior and helping people make good decisions. There are many blog posts yet to be written about checks and balances in the design of measurement processes. But it’s a good idea to begin with a shared vocabulary of measurement. Two fundamental terms in the measurement discipline are accuracy and precision.
A bag full of marbles may be said to contain ‘about 100 marbles,’ but it is accurate to actually count the marbles and be able to say that the bag has ‘exactly 103 marbles.’ With the marbles, we reach absolute accuracy by direct observation. But most measures can’t be directly observed, and determining their accuracy is less straightforward. When a web site says that it is 91 degrees where I live, but my outdoor thermometer says that it is 86 digress, it isn’t clear which measurement is more accurate, especially since temperature isn’t observable. It is the measurement process that determines accuracy.
Observability doesn’t guarantee accuracy. A call center manager may measure call volume per employee during a one-hour period by watching his employees and making tick marks on a piece of paper, but that direct observation is subject to error. Statistics from the telephone equipment in the call center should be more accurate (unless there is a flaw in the equipment’s software).
We’re surrounded by measurements that we can’t independently confirm, and we take the accuracy of most measurements for granted. Our inclination to trust a measure has as much to do with the source of the information (the web site vs. my wall thermometer) as the measurement process itself. It’s also important to remember in the business setting that those performing a measurement process may have an interest in the interpretation of that measurement.
Precision is a simple attribute of measurement. It is simply the fineness of distinctions made when expressing a variable. When I tell you that the bag of marbles contains ‘about 100 marbles,’ I am fairly representing an absence of precision in the measurement. But if I tell you that the bag contains ‘108 marbles,’ I am expressing a degree of precision (down to the single marble) that implies accuracy, that is independent of how I actually arrived at the number. If the call center manager’s tick marks result in a calculation of 3.42 calls per employee per hour, we trust his measure simply because of that degree of precision.
It is simple human nature to confuse precision with accuracy. My outdoor thermometer is digital, and is actually telling me that it is 86.9 degrees outside. The web site says 91 degrees. If I hadn’t taken the time to think about it, I might be more inclined to believe the outdoor thermometer, just because of that decimal point, but my healthy skepticism says that I shouldn’t. All I know for sure is that it’s hot.