This series of posts (in three parts) is adapted from an article of the same name that appeared in Harvard Business Publishing’s Balanced Scorecard Report in 2006.
In Part I, I asserted that perception matters very much to the strategy of an organization. Perception of external stakeholders, of customers, and of employees. Often, the change program requires measurements of customer and employee perceptions. Here, we consider how organizations go about gathering these perceptions, which is a key factor in the success of the change program.
Ensuring Survey Success: Skillful Research Design is Vital
A survey program is the best way to regularly monitor stakeholder perceptions. E-mail and Web-based survey tools enable faster design, execution, and analysis, and have reduced the cost considerably. Many enterprises already have e-mail address lists from the Web sites and customer databases they maintain for direct communication and marketing purposes. Wireless telephony and text messaging enable nearly real-time data collection and analysis. Technology, however, is no substitute for good research design, and in amateur hands, such tools amplify the risk of getting unactionable results or even causing adverse consequences.
Regrettably, to save money, many firms opt for do-it-yourself research design, even when they lack the expertise to do so effectively. The principles of research design are beyond the scope of this article; suffice it to say that unless you have a professional market research staff in house, do-it-yourself research design is risky, unlikely to yield much useful information, and can actually do harm. The “Hawthorne Effect” demonstrated that the very act of studying a group of subjects changes members’ behavior. A poorly designed survey can antagonize the subjects, or, if they fear repercussions from any negative responses, discourage candor. Either way, it can taint results.
The research designer must be both unbiased about the research topic and familiar with the study environment. Unwittingly or not, do-it-yourselfers often allow their leaders’ biases to creep into the questions. But subtle changes in the way a question is phrased—even the order of the questions—can dramatically affect how people respond. Allow leaders to contribute their opinions when establishing hypotheses, but keep them a safe distance from the actual design.
Strategy-driven research begins with the cause-and- effect relationships among strategic objectives on a strategy map. Strategic objectives for customers and employees require perception measures to give a full picture of performance. While financial measures like sales volume or market share reflect the results of customer behavior, leading measures of customer perception can help executives anticipate changes in these ultimate outcome measures.
Changing people’s perceptions is often central to strategy. Well-designed research might provide leaders with insights about which perceptions need to change or help test hypotheses about how a change in perception will actually occur. A customer perception measure can validate the effect of an internal process initiative.
For example, as part of its strategy to develop more profitable relationships with its customers, a bank undertakes an initiative to have customer service agents spend more time on the telephone with customers calling for support. By equipping agents with better customer information, the bank believes they will be able to probe beyond the original scope of each call and promote additional products appropriate to the caller. A sound customer feedback program can enable the bank to validate the assumptions underlying its strategy, or to uncover negative perceptions (e.g., that calls are taking too long or agents’ questions are intrusive). Only by measuring agents’ new behavior (for example, through average call duration), along with agent and customer perceptions, can increased sales reasonably attributed to the new strategy.
While an inexperienced research designer would simply begin by writing questions, a seasoned designer would start by studying the environment and its population groups, as well as the organization’s strategy, to decide the best method (e.g., surveys, focus groups, or data mining) and frequency, and how to segment the research population. Drawing on her academic and practical understanding of human behavior, statistical analysis, and survey design, she would then develop a comprehensive research plan and plan follow-on research.
Conquer Survey Fatigue
Leaders seeking “silver bullets” for intractable problems don’t often have the patience to execute surveys repeatedly. But few surveys provide full value in one execution. Periodic sampling of a population reveals trends that are impossible to see in a single snapshot. Often the absolute value of a perception measure is meaningless; the insight comes from tracking the direction and magnitude of change in the measure over time.
“Survey fatigue” is a common reason why organizations resist using surveys. Employees in firms with Internet survey tools may be bombarded with narrow, one-off surveys from multiple sources whose timing is not coordinated and that, over time, do little more than discourage response. Such surveys may be sent to every member of a large population when a relatively small, carefully chosen representative sample would be valid—thereby reducing survey frequency and minimizing fatigue. All research requests should be funneled through a single coordinator.
Rather than surveying the entire employee population annually, the organization could send the same basic survey to a controlled random sample every quarter (25% of the employee population), polling different subgroups each time. The sample size is determined mathematically, according to population size and acceptable margin of error. A basic set of questions could be asked each time, and carefully targeted new questions could be included when needed to understand new challenges. Under this model, no employee in the target group would receive more than one survey per year. By generating quarterly findings (rather than the usual annual ones), this model is especially appropriate for strategy reporting.
Next: Alternatives to Surveys