In our pursuit of a shared vocabulary of measurement, we’ve already considered the ideas of accuracy, precision, and healthy skepticism. Here, we take a step back and look at key terms of measurement concept, dimension, and indicator.
A measurement concept is a mental image that describes an area of interest, such as speed, warmth, or comfort. The key to thinking about a concept is that it springs from an idea; an impression or perception that cannot be directly measured. To conceptualize an idea is to specify what we mean when we use that mental image.
Imagine that you’re an airline executive interested in improving passenger comfort. To say that I was ‘comfortable’ on a recent flight is to share my perception, but your understanding of my experience is not quantifiable. When you ask me what I mean by ‘comfort,’ I might say that I had a good seat, and that the flight attendant was helpful. In this example, ‘good seat’ is one of my concepts of comfort. But the concept of a ‘good seat’ has different meanings to different people. I describe a ‘good seat’ as an aisle seat with ample legroom and width, which are some of my dimensions of the concept (another has to do with being next to an empty seat).
In the course of speaking to many other passengers, and your understanding of possible approaches to improving passenger comfort, you conclude that the best leading indicator of the dimension of comfort is whether the passenger is in his or her preferred seat type (since some prefer window or even middle seats). So you choose ‘share of selected passengers in their preferred seat type’ as your indicator of the dimension of passenger comfort. Of course, you could have chosen other dimensions, and other indicators, but you know that after a point, the measurement process is not improved by more measures.
As part of the airline’s initiative to improve passenger comfort, you identify a target group of passengers (perhaps those at or above a loyalty program tier), and capture their preferred seat type in their online profile. You then modify seat assignment policies and processes (both automated and manual) to try to increase the number of times that those selected passengers are seated according to their preference. Only now have we arrived at a quantifiable, leading indicator of passenger comfort. The hypothesis implied by this choice is that getting passengers in their preferred seats will cause them to experience comfortable flights more often. An appropriate lagging indicator would be a survey of those selected passengers to capture their actual experience of comfort, and to then correlate that data with their seat assignments.
Of course, this example does not show how measures are usually selected. Measurement is often difficult, and every measurement choice implies a hypothesis. Leadership teams tend to choose those measures that are easily captured, not those that are reasoned choices as shown in the example. The choice of measurement may be delegated to subordinates whose understanding of the strategy may be incomplete, or have a vested interest in presenting a measure whose outcomes can easily be, ahem, ‘managed.’
A shared understanding of the vocabulary of measurement will enable leaders to more effectively consider and chose measures that will motive desired change behaviors and effective describe the performance and impact of strategic initiatives.
Please share below some of the best and worst measures in your change program.