Tag Archive

metrics

Gaming metrics

When I suggest that people will game whatever metrics we put in place, I’m often met with shocked indignation. We would never game the numbers! And yet we do.

Prioritization

There two different times that we need to prioritize work and we should be using completely different approaches to that prioritization, for each stage.

Wait states

The metrics covered in our Flow Metrics Basics class show how to measure different aspects of flow, but those metrics don’t in themselves, show how to improve the flow. We’ll look at one way to improve now.

Basic Flow Metrics

Flow metrics tell us about the activity in the system; how well is the work flowing. These are often the first metrics that I consider when trying to improve a system. I start here partially because these metrics are easier to collect than many others, and because I can still make many effective decisions based on just this.

Visualizing Flow Efficiency

I’m playing around with visualizing flow efficiency in my JiraMetrics tool. Flow efficiency is the percentage of time that we’re actually adding value to the work item divided by the total time. So if a ticket is open for 10 hours but in that time we only spend 2 hours actually working on it then the flow efficiency would be 2 / 10 or 20%.

Tracking metrics

Most companies I work with have a desire to track metrics for their development teams, and I support this. It’s hard to improve something we can’t see so metrics are a good first step as we seek to improve.

Flow Efficiency

Flow efficiency (sometimes called Cycle Efficiency) is a metric that gives us a sense of how much time work is waiting. A flow efficiency of 100% would indicate that we are adding value to the work item for the entire time it’s in progress. 50% would imply that half the time we’re working on it and half the time the work is just waiting.

Who should look at what metrics?

When we talk about metrics, there is often an assumption that everyone in the company needs the same data to make decisions and this is dangerously incorrect. Different levels of the organization need different kinds of data to make effective decisions. Yet, all too often we use the wrong data at the wrong point.

Technical Debt

The term “technical debt” is widely used in the industry even if there isn’t a clear definition of it and almost nobody uses the term in the way Ward Cunningham meant when he first coined it. It’s most commonly used to describe things in our environment, usually but not always code, that slow us down. These are things that are working - not bugs - but that are implemented in a poor way that makes them more difficult to understand or modify.

One Thing vs Multiple Things

When creating a forecast first ask yourself whether you are forecasting One Thing or Multiple Things. It’s not always clear which of these situations you are in but the approach you take to creating the forecast will differ significantly. This post will help you to figure out which approach to take.

Improving Predictability - Average Age

In a previous post I’ve introduced the four assumptions behind Little’s Law and discussed the first two assumptions in detail. If you haven’t read those previous posts I encourage you to go back to understand the background. As a reminder, the four assumptions are listed below.

Improving Predictability - All work must finish

In a previous post I introduced the four assumptions behind Little’s Law and the idea that they are critical to understanding and improving your system’s predictability. We’ve also already discussed the first assumption regarding the equality of average arrival and departure rates. If you haven’t read those previous posts I encourage you to go back to understand the background. As a reminder, the four assumptions are listed below.

Improving Predictability - Average Arrival and Departure Rates

In a previous post I introduced the four assumptions behind Little’s Law and the idea that they are critical to understanding and improving your system’s predictability. If you haven’t read that post I encourage you to go back to understand the background. As a reminder, the four assumptions are listed below.

Improving Predictability

Little’s Law is an equation that frequently appears in discussions of Kanban systems. While initially formulated as a part of queuing theory to describe the length of time people would spend in stores it has since been applied to many other contexts from manufacturing to knowledge work (particularly Kanban for the purposes of today’s conversation).

Determining cycle time from an online system

This post is aimed at a fairly niche audience so if you aren’t trying to make sense of poor data out of some ticketing system then you might want to skip this one.

Improve the work, not the metrics

One of the key practices supporting continuous improvement is making your work, and how you do the work, visible. This starts by tracking the progress of that work in a highly visual way, often by using a kanban board. Once that work is being tracked we can begin to gather that data and start to gain insights into where our biggest opportunities for improvement are, often by using the metrics defined in The Three Flow Metrics (Plus One).

The three flow metrics (plus one)

Some of the best indicators of team performance are the flow of both new information into the team and of value out of the team. If we can improve visibility into these indicators, and therefore the opportunities for the team to improve the way they work, it becomes possible for the team to support their organization in ways they couldn’t before. There are three standard metrics that are core to understanding the effectiveness of any flow-based system. The relationship between the three metrics is defined by Little’s Law. When applied to the systems used to enable knowledge work the law is usually restated in terms of Throughput, Work In Progress (WIP), and Cycle Time.

Back to Top ↑

waste

Prioritization

There two different times that we need to prioritize work and we should be using completely different approaches to that prioritization, for each stage.

Wait states

The metrics covered in our Flow Metrics Basics class show how to measure different aspects of flow, but those metrics don’t in themselves, show how to improve the flow. We’ll look at one way to improve now.

Continuous improvement

Back in the days when faxing between companies was a popular thing, I recall a client that had a workflow like this:

  1. Fax arrives and is printed by the fax machine
  2. Paper is picked up by a person and carried to the scanner where is it digitized.
  3. Paper is immediately shredded because there was confidential information on it.

Fractional people

I worked with two teams that shared a database specialist, for a fairly proprietary system. Because she was on two teams, they had her attend all meetings for both teams. Two standups, two planning meetings, two retrospectives. She ended up doing crazy amounts of overtime because she couldn’t get her regular work done during the day. Her days were filled with meetings, and it was only after everyone else left that she got anything done.

Improving meetings

When we look for opportunities for improvement, at some point every team will bring up meetings as being an ongoing problem for them. When we dig into what the actual problems are, we find they always fall into one of three categories:

Per-story estimates

Per-story estimates were an interesting experiment that failed and it’s time to move on. Today, we have better ways so it’s time to stop putting individual estimates on stories. This is equally true for Scrum and Kanban teams.

Technical Debt

The term “technical debt” is widely used in the industry even if there isn’t a clear definition of it and almost nobody uses the term in the way Ward Cunningham meant when he first coined it. It’s most commonly used to describe things in our environment, usually but not always code, that slow us down. These are things that are working - not bugs - but that are implemented in a poor way that makes them more difficult to understand or modify.

The cost of interruptions and how to reduce it

Interruptions can be a significant source of waste. By their nature, interruptions cause a context switch as we lose track of what we had been working on to focus on the interruption. There is a significant cost to that context switch as it takes time and effort away from the task at hand. There is also a real impact on quality as mistakes are far more likely to happen when we’re distracted.

Waste: Psychological Distress

One of the more subtle forms of waste is psychological distress. When we are afraid or anxious, our sympathetic nervous system activates to prepare us for one of fight, flight or freeze. All good responses in a survival situation.

Understanding waste in the system

In Kanban, we are always trying to optimize for efficiency, effectiveness and predictability. Waste in the system is something that hurts all three of these objectives and is something we want to remove or reduce wherever possible.

Lowering the Water Level - the metaphor explained.

Toyota has this metaphor of “lowering the water level” that they use when looking for opportunities for improvement. This video explains the metaphor and how it applies to your Kanban system.

Back to Top ↑

predictability

Slicing stories

In an agile environment, we split our work down into what we call “stories”, that are the smallest unit of value passing through a workstream. Unfortunately, we have a tendency to over-complicate story writing, making it unnecessarily hard. Done well, it can be a simple process of taking small steps, repeatedly.

Keeping people busy

The Kanban Guide talks about optimizing the workflow for three different attributes: effectiveness, efficiency, and predictability. It talks about the fact that any optimizations we perform will be a balance across these three and that over-optimizing on one may make the others worse.

Steps to improving predictability

If you have a need to know when the work will be done or how much you can do in a certain period of time then predictability will be important to you. We have great tools like Monte Carlo for probabilistic forecasting but the truth is that the forecast we generate is only as good as the data we give it. Garbage in yields garbage out. So how do we improve our data to make it inherently more predictable?

One Thing vs Multiple Things

When creating a forecast first ask yourself whether you are forecasting One Thing or Multiple Things. It’s not always clear which of these situations you are in but the approach you take to creating the forecast will differ significantly. This post will help you to figure out which approach to take.

Improving Predictability - Average Age

In a previous post I’ve introduced the four assumptions behind Little’s Law and discussed the first two assumptions in detail. If you haven’t read those previous posts I encourage you to go back to understand the background. As a reminder, the four assumptions are listed below.

Improving Predictability - All work must finish

In a previous post I introduced the four assumptions behind Little’s Law and the idea that they are critical to understanding and improving your system’s predictability. We’ve also already discussed the first assumption regarding the equality of average arrival and departure rates. If you haven’t read those previous posts I encourage you to go back to understand the background. As a reminder, the four assumptions are listed below.

Improving Predictability - Average Arrival and Departure Rates

In a previous post I introduced the four assumptions behind Little’s Law and the idea that they are critical to understanding and improving your system’s predictability. If you haven’t read that post I encourage you to go back to understand the background. As a reminder, the four assumptions are listed below.

Improving Predictability

Little’s Law is an equation that frequently appears in discussions of Kanban systems. While initially formulated as a part of queuing theory to describe the length of time people would spend in stores it has since been applied to many other contexts from manufacturing to knowledge work (particularly Kanban for the purposes of today’s conversation).

Back to Top ↑

forecasting

Driving to the airport

Imagine we wanted to estimate how long it would take to drive to the airport. You might see that it’s 50km to the airport and that your car can drive at 100km/hour. Therefore it will take 30 minutes, right?

Monte Carlo under the covers

Monte Carlo forecasting is the most common form of probabilistic forecasting that we see. It’s compelling because it can provide a highly accurate forecast of when work will be done, with relatively little effort.

What is Probabilistic Forecasting?

Do your customers ever ask “When will it be done?” When dealing with the future, there’s almost never an accurate deterministic answer (Tuesday, exactly at 3:45pm) to that question but there is an accurate probabilistic answer (85% chance of completion on or before October 1) and in most cases, it’s a lot easier to calculate than you’d expect.

Reference class forecasting

Weather predictions are probabilistic, not deterministic. That means there isn’t a single right answer that we can calculate. We can’t say it will rain at 11:05 but we can say that there’s an 80% chance of rain today. Forecasting when we’ll be done is also probabilistic, in exactly the same way. We can say based on past throughput data that we have an 85% chance of being done on or before May 12.

Per-story estimates

Per-story estimates were an interesting experiment that failed and it’s time to move on. Today, we have better ways so it’s time to stop putting individual estimates on stories. This is equally true for Scrum and Kanban teams.

What is a Service Level Expectation?

A service level expectation is a probabilistic forecast of how long it will take a single item to pass through the system. For example: “85% chance of completing in four days or less”.

Steps to improving predictability

If you have a need to know when the work will be done or how much you can do in a certain period of time then predictability will be important to you. We have great tools like Monte Carlo for probabilistic forecasting but the truth is that the forecast we generate is only as good as the data we give it. Garbage in yields garbage out. So how do we improve our data to make it inherently more predictable?

One Thing vs Multiple Things

When creating a forecast first ask yourself whether you are forecasting One Thing or Multiple Things. It’s not always clear which of these situations you are in but the approach you take to creating the forecast will differ significantly. This post will help you to figure out which approach to take.

Back to Top ↑

flow

Prioritization

There two different times that we need to prioritize work and we should be using completely different approaches to that prioritization, for each stage.

Wait states

The metrics covered in our Flow Metrics Basics class show how to measure different aspects of flow, but those metrics don’t in themselves, show how to improve the flow. We’ll look at one way to improve now.

Basic Flow Metrics

Flow metrics tell us about the activity in the system; how well is the work flowing. These are often the first metrics that I consider when trying to improve a system. I start here partially because these metrics are easier to collect than many others, and because I can still make many effective decisions based on just this.

Visualizing Flow Efficiency

I’m playing around with visualizing flow efficiency in my JiraMetrics tool. Flow efficiency is the percentage of time that we’re actually adding value to the work item divided by the total time. So if a ticket is open for 10 hours but in that time we only spend 2 hours actually working on it then the flow efficiency would be 2 / 10 or 20%.

Fractional people

I worked with two teams that shared a database specialist, for a fairly proprietary system. Because she was on two teams, they had her attend all meetings for both teams. Two standups, two planning meetings, two retrospectives. She ended up doing crazy amounts of overtime because she couldn’t get her regular work done during the day. Her days were filled with meetings, and it was only after everyone else left that she got anything done.

Flow Efficiency

Flow efficiency (sometimes called Cycle Efficiency) is a metric that gives us a sense of how much time work is waiting. A flow efficiency of 100% would indicate that we are adding value to the work item for the entire time it’s in progress. 50% would imply that half the time we’re working on it and half the time the work is just waiting.

Back to Top ↑

improvement

Playing the long game

If I was planning to disband my team this week and I only cared about what they could deliver in a couple of days then I’d be focused on making sure that everyone was doing that piece of work that they were most skilled at.

Continuous improvement

Back in the days when faxing between companies was a popular thing, I recall a client that had a workflow like this:

  1. Fax arrives and is printed by the fax machine
  2. Paper is picked up by a person and carried to the scanner where is it digitized.
  3. Paper is immediately shredded because there was confidential information on it.

Looking for improvement

I was asked recently what things I’d look at to determine if a team or group is improving and there are three main areas. In all three cases, none of these prove that improvement is happening. What they do provide is a place to me to start asking questions so that I can discover more.

Excessive sub-tasks

I worked with a team once that had gotten into the habit of creating sub-tasks for every little thing they could think of. This list included such obvious items as “write the code” and “test the code” and a typical story would have about twenty of these sub-tasks attached.

Premature optimization

We have a tendency to think that making any one part of the workflow more efficient will make the overall workflow also more efficient and that’s just not true. Part of that is that not all parts of the workflow are on the critical path and improving something that isn’t currently a bottleneck won’t help the overall flow. But there’s a second reason that’s less obvious - sometimes optimizing for simple cases in the workflow, can make other parts of the workflow much worse.

“We tried Kanban and it didn’t work”

I sometimes run across teams that say “we tried kanban and it didn’t work”. When I hear this, I’m always genuinely curious and ask for more details about what they’d done and what specifically didn’t work for them.

Improve the work, not the metrics

One of the key practices supporting continuous improvement is making your work, and how you do the work, visible. This starts by tracking the progress of that work in a highly visual way, often by using a kanban board. Once that work is being tracked we can begin to gather that data and start to gain insights into where our biggest opportunities for improvement are, often by using the metrics defined in The Three Flow Metrics (Plus One).

Back to Top ↑

WIP

WIP by Parent

One of the charts built into JiraMetrics is WIP by Parent, as shown below. What this shows is the total work in progress (WIP) on each given day. The WIP is then grouped by colour according to the parent (Epic in this case) that the original ticket belonged to.

High WIP invalidates prioritization

The more items we have in progress at once (WIP), the less important our initial prioritization is. When we work on one item at a time then items get done in the order we started them so we are completing them in the order of most importance.

Massively overburdened with WIP

About once a year I run across a team that has at least ten times as many items on the board as there are people on the team. The worst I’ve ever seen was a team of ten people with 227 items in progress.

Setting initial WIP limits

When starting a team up with Kanban, one of the earliest questions is how to you set initial WIP limits. The simple rules we use are covered in this video.

Lowering the Water Level - the metaphor explained.

Toyota has this metaphor of “lowering the water level” that they use when looking for opportunities for improvement. This video explains the metaphor and how it applies to your Kanban system.

Back to Top ↑

standup

The facilitators role

If you’re facilitating the daily coordination meeting (standup, daily scrum, whatever you want to call it), and you’re doing all the talking, then you’re doing it wrong.

Jira’s Start Standup Button

For a while now, I’ve been noticing a “start standup” button at the top of Jira boards and I’ve been wondering what it did. Today I pushed that button in the hope that it would do something to help make the standup more effective, and now I wish I hadn’t.

Intent of standups / scrum meetings

The intent of a team standup/scrum meeting is to resynchonize on the work in progress and decide how we, as a team, are going to complete it. If your standup is horrible, then focus on the intent, not the format you’ve been doing. What would you have to change to make the meeting useful?

Improving the daily coordination meeting

Just about all agile methods have some kind of a daily coordination meeting. It might be called a standup or a daily coordination meeting or a daily scrum or any number of other things. The point is that this meeting is focused on actively managing the work and it’s frequently done poorly.

Back to Top ↑

value

Building the right thing

When I’m working with a development team, I’ll often start by having them walk me through what business need they’re solving for. I’m less interested in the code or the architecture until I’ve first understood what problem we’re trying to solve.

Focus on Value

Whatever we focus on, we’ll get more of. If we focus on the flow of value then we’ll get more value. If we focus on being busy then we’ll get more of that.

Understanding the users

My very first professional programming job was at a large bank that had invested in a full UX lab. One-way glass, cameras, and a full setup where we could watch real branch staff using the software we had been working on.

When we do everything right and it still doesn’t solve the right problem

The Choluteca Bridge, in Honduras, was built in an area known for hurricanes and other strong weather. It was designed to withstand the destructive force of a hurricane so when hurricane Mitch came through later that same year, it was no surprise that the bridge sustained only minor damage. Clearly, it had been designed and implemented well.

Back to Top ↑

meetings

The facilitators role

If you’re facilitating the daily coordination meeting (standup, daily scrum, whatever you want to call it), and you’re doing all the talking, then you’re doing it wrong.

Improving meetings

When we look for opportunities for improvement, at some point every team will bring up meetings as being an ongoing problem for them. When we dig into what the actual problems are, we find they always fall into one of three categories:

Improving the daily coordination meeting

Just about all agile methods have some kind of a daily coordination meeting. It might be called a standup or a daily coordination meeting or a daily scrum or any number of other things. The point is that this meeting is focused on actively managing the work and it’s frequently done poorly.

Back to Top ↑

stories

Slicing epics

We talk a lot about slicing stories but then when it comes to slicing larger types (epics, features, etc), we tend to wave our hands and say “it’s the same, only bigger”, which while true, is rarely helpful.

Slicing stories

In an agile environment, we split our work down into what we call “stories”, that are the smallest unit of value passing through a workstream. Unfortunately, we have a tendency to over-complicate story writing, making it unnecessarily hard. Done well, it can be a simple process of taking small steps, repeatedly.

Per-story estimates

Per-story estimates were an interesting experiment that failed and it’s time to move on. Today, we have better ways so it’s time to stop putting individual estimates on stories. This is equally true for Scrum and Kanban teams.

Back to Top ↑

collaboration

Playing the long game

If I was planning to disband my team this week and I only cared about what they could deliver in a couple of days then I’d be focused on making sure that everyone was doing that piece of work that they were most skilled at.

Self-sufficient teams

A number of years ago, a client of mine had a new feature request come to one part of the company. It was a significant change that would touch quite a few teams to get done and when they asked those teams how long it would take, the answer was somewhere between 10 and 12 weeks.

Back to Top ↑

prioritization

High WIP invalidates prioritization

The more items we have in progress at once (WIP), the less important our initial prioritization is. When we work on one item at a time then items get done in the order we started them so we are completing them in the order of most importance.

When everything is a priority

I was asked what to do when the customer won’t prioritize the stories and insists that everything has to be done now. That brought back memories of the first time that had happened to me. This was a very long time ago and I can’t remember who had given me the idea to approach it this way.

Back to Top ↑

retrospective

The facilitators role

If you’re facilitating the daily coordination meeting (standup, daily scrum, whatever you want to call it), and you’re doing all the talking, then you’re doing it wrong.

Back to Top ↑

risk

Risk Management

Yesterday I went on a guided hike to teach people how to safely hike through bear country. Specifically grizzy bear country. Unfortunately, we didn’t see any bears, but we did learn an amazing amount about them, and saw some spectacular glaciers and alpine terrain.

Back to Top ↑