"Perhaps what you measure is what you get. More likely, what you measure is all you’ll get. What you don’t (or can’t) measure is lost" - H. Thomas Johnson
I think the above two quotes really sum up the “management by numbers” debate well in that metrics or Key Performance Indicators (KPI’s) will drive behaviour but not necessarily the ones you want or the ones that will help sustain your business.
Complexity
The way complex systems interact is often difficult to understand and can be the bane of business. When we don’t understand something we create hypotheses. And when those hypotheses prove to be correct it’s a wondrous thing. Where it gets tricky is when it can’t be or isn’t proven, and the varying opinions become the basis for non-productive conflict or delusion.
There are a couple of ways to react to this. One: You can accept that things are too complex to understand and give up. The problem with that approach is that it’s not productive. Two: You can try to make improvements and to understand what’s going on despite the complexity. I’m a big proponent of putting in effort to improve despite the fact that you may fail. You will at least have been proactive and the odds are, if you do it enough, you’ll at least improve and potentially succeed. The question I struggle with is how many times should you try and at what point should you give up? Obviously you’ll stop when your resources run out but is that the “best” time?
Metrics
I can think of lots of examples where metrics failed. Where specifications were set and met and yet there were problems. One case was a Tier 2 supplier who wanted to maximize their profits and really didn’t understand the application. Their cost cutting measure still met the specifications but not the intended purpose of use, resulting in premature failures. The resulting warranty costs definitely outweighed any cost savings not to mention the cost of damage to everyone’s reputation.
It’s possible that they didn’t care about potentially negative consequences but I seriously doubt it given how poorly it reflected on their competence and credibility. It also reflected badly on the OEM (Original Equipment Manufacturer) and Tier 1 supplier since it could be argued that the specification was inadequate.
There are ways to approach complexity to develop understanding but it’s helpful to put metrics into context first.
Key Performance Indicators (KPI’s)
Source Merriam Webster http://www.merriam-webster.com
• Key – 3 key adj Definition of KEY: important, fundamental
• Performance - noun 4b: the manner in which a mechanism performs
• Indicator – noun 1b: gauge. Example of INDICATOR: Economic indicators suggest that prices will go up.
In business, the term “Key Performance Indicator’s” provides a better context than “metrics”. Metrics shouldn’t be the-end all, be-all; they’re simply indicators of performance although they should be key ones. For example, as the dashboards in our vehicles demonstrate we need gas in order to drive but we don’t want to confuse the requirement for gas with the purpose of getting us to a destination. Fuel level is a key indicator for vehicle operation but it becomes irrelevant if we decide to use some other method of travel.
Driving Behaviour
Temple Grandin is a fascinating person and author. She’s autistic and has a Doctorate in Animal Science. She’s very passionate about the humane treatment of animals and has developed processes and methods to get cattle (and other animals) to calmly go to slaughter. Her challenge was on the human end; despite significant effort, slaughter house personnel kept relapsing back to old habits of using cattle prods, etc. Undaunted, in typical Temple style, she decided that if she could figure out how to get cattle to quietly go to slaughter, then she could figure out how to get humans to co-operate too. What she discovered was that auditing worked, i.e. when the desired behaviours were measured, they were maintained.
Similarly, providing bio-feedback can lead to amazing changes. Measuring also works effectively in other contexts too although you could certainly argue that it doesn’t always work given America’s struggle with obesity and obsession with weight loss. Clearly some things are harder to change than others. In the context of business however, we rely on metrics to drive quality improvements. Unfortunately, problems can arise in how we tend to think of “measure”. For example, I like the Hauser & Katz saying that, “Being vaguely right is better than being precisely wrong”.1 Their assertion is that measuring something precisely or easily isn’t as important as what you measure. What matters is that the measurement be pertinent. Things like culture, identity and emotion can have a big impact in business and yet it’s unlikely you’ll be able to precisely measure it, although, you might be able to differentiate between a supportive or non- supportive culture.
Other Considerations
I think it’s interesting that in Chip & Dan Heath’s book, “Switch” they don’t mention “metrics” as a change mechanism. Instead they talk about concepts like following the bright spots which I think of as leveraging benchmarks or figuring out how to apply what’s worked previously in new areas.
Their analogy reminded me of Six Sigma and Shainin techniques as the process is similar. Identify the issue, develop a hypothesis on the root cause, prove the root cause using paired comparisons, etc. Then develop strategies to repeat the desired outcome and minimize the variation. Notice that it takes time, analysis and perseverance. These types of improvements are responsible for the great quality we see in automotive vehicles these days but it’s not cheap and maybe that’s the rub. Understanding the complexity is difficult and resolving things that are difficult is typically expensive - but it can be done. Unfortunately, hiring a Six Sigma Engineer may be out reach of many small businesses but there are alternatives. (I recommend reading, “Switch”, to get some good ideas.)
Another thing to consider is that there are frequently conflicting perspectives; and it’s about striking an appropriate balance or finding the sweet spots. My favourite example is balancing between the cost of production and design optimization. It’s always nice to over-deliver but there’s little point in producing a Taj Mahal when a cottage will do, especially if you’re on a limited budget. Using targets to support an appropriate “balance” can help with understanding of what makes sense and can assist in providing context and perspective.
The Heath brothers also make the point that there are times when you don’t want to use measurement because people (being the lemmings we are) will follow the social proof or in their words: they’ll “follow the herd” or whatever the majority are doing. That is, you don’t want to show or reinforce what the majority are doing if it’s the wrong thing.
Metrics prove change. But what comes first: the chicken or egg? You can argue that the methods used to change behaviour can also be used to achieve the metrics. Regardless, you need to be working towards a purpose so if what you were doing isn’t working then you need to change and verify it achieved a positive outcome.
The bottom line? Don’t waste your time (and money) measuring something because it’s easy. Measure it because it supports the purpose of the business and remember, as is the case with vehicle dashboards: “metrics” (in this instance a fuel gauge) can have little to do with the purpose of getting to a destination and yet still be very useful in ensuring the “success” of the journey.
1 From: Metrics: You Are What You Measure! by John R. Hauser (MIT Prof) and Gerald M. Katz (Executive Vice President, Applied Marketing Science, Inc also went to MIT)
April 1998 http://www.mit.edu/~hauser/Papers/Hauser-Katz%20Measure%2004-98.pdf
