I frequently talk to clients about metrics, and there is usually an understandable desire to measure too much. “Let’s measure these fifty things everyday so that we know if everything is ok.”
Are you really going to look at fifty of those on a regular basis? You might for the first few days or even weeks, and then you won’t anymore.
I had an executive once ask me to email a freshly generated 250 page report every hour through the day to his cell phone, so he could keep an eye on things. It took far too long to convince him that this was not a good use of anyone’s time and that in any case, it was giving him entirely the wrong level of detail.
At the opposite end, I had one client that had automated six key metrics to be sent to all senior leaders every 45 minutes. Six simple numbers from which they could tell at a glance if everything was running smoothly or if they needed to refocus their attention.
Another client had a large wall of metrics and analytics on monitors beside the exit. Anyone leaving the building could see at a glance if everything was ok or if there was something needing their attention before they left.
These are all examples of realtime, or near realtime, metrics but not everything has to be that fast. If we’re looking at the speed at which a software development team is completing features, looking at metrics even once a week might be too often.
If we’re looking at value delivery and whether we built the right things for our customers, even quarterly might be too often.
So we need to consider not only what we’re measuring but also how frequently we review that data.
At a high level, there are three main groupings of metrics that I normally want to start with.
Flow metrics tell us if the work is moving through the system. I usually start here, partially because they’re the easiest to measure, and partially because they will uncover a huge number of the problems with little effort. If the work isn’t getting done then really nothing else matters.
Value metrics tell us if we’re spending our money wisely. Are we doing the right work? Are we doing it at the right time? Is it of the right quality?
Sustainability metrics tell us if we can continue at this rate indefinitely or if we’re just running towards a cliff and going to crash.
I tend to look at flow metrics regularly as they give me immediate feedback on the health of the system. Value and sustainability tend to be longer term views into the system. They take longer to collect and are reviewed less often.
What do I almost never care about? How busy we are. Busyness is not a proxy for either effectiveness or efficiency, and I’ve written about that before so I’ll leave it today.
So I start with “is the work moving well?”, followed by “is the the right work?”, and finally “is what we’re doing sustainable?”
For each of these, I want to find a small set of metrics that give me a glimpse into the system. I don’t want too many different measurements as that becomes overwhelming and I don’t want to have too few because then they’re easily gamed.
Not sure what you should be measuring in your environment? Let’s talk.