Tag Archive

metrics

Basic Flow Metrics

Flow metrics tell us about the activity in the system; how well is the work flowing. These are often the first metrics that I consider when trying to improve a system. I start here partially because these metrics are easier to collect than many others, and because I can still make many effective decisions based on just this.

Tracking metrics

Most companies I work with have a desire to track metrics for their development teams, and I support this. It’s hard to improve something we can’t see so metrics are a good first step as we seek to improve.

Who should look at what metrics?

When we talk about metrics, there is often an assumption that everyone in the company needs the same data to make decisions and this is dangerously incorrect. Different levels of the organization need different kinds of data to make effective decisions. Yet, all too often we use the wrong data at the wrong point.

Technical Debt

The term “technical debt” is widely used in the industry even if there isn’t a clear definition of it and almost nobody uses the term in the way Ward Cunningham meant when he first coined it. It’s most commonly used to describe things in our environment, usually but not always code, that slow us down. These are things that are working - not bugs - but that are implemented in a poor way that makes them more difficult to understand or modify.

One Thing vs Multiple Things

When creating a forecast first ask yourself whether you are forecasting One Thing or Multiple Things. It’s not always clear which of these situations you are in but the approach you take to creating the forecast will differ significantly. This post will help you to figure out which approach to take.

Improving Predictability - Average Age

In a previous post I’ve introduced the four assumptions behind Little’s Law and discussed the first two assumptions in detail. If you haven’t read those previous posts I encourage you to go back to understand the background. As a reminder, the four assumptions are listed below.

Improving Predictability - All work must finish

In a previous post I introduced the four assumptions behind Little’s Law and the idea that they are critical to understanding and improving your system’s predictability. We’ve also already discussed the first assumption regarding the equality of average arrival and departure rates. If you haven’t read those previous posts I encourage you to go back to understand the background. As a reminder, the four assumptions are listed below.

Improving Predictability - Average Arrival and Departure Rates

In a previous post I introduced the four assumptions behind Little’s Law and the idea that they are critical to understanding and improving your system’s predictability. If you haven’t read that post I encourage you to go back to understand the background. As a reminder, the four assumptions are listed below.

Improving Predictability

Little’s Law is an equation that frequently appears in discussions of Kanban systems. While initially formulated as a part of queuing theory to describe the length of time people would spend in stores it has since been applied to many other contexts from manufacturing to knowledge work (particularly Kanban for the purposes of today’s conversation).

Determining cycle time from an online system

This post is aimed at a fairly niche audience so if you aren’t trying to make sense of poor data out of some ticketing system then you might want to skip this one.

Improve the work, not the metrics

One of the key practices supporting continuous improvement is making your work, and how you do the work, visible. This starts by tracking the progress of that work in a highly visual way, often by using a kanban board. Once that work is being tracked we can begin to gather that data and start to gain insights into where our biggest opportunities for improvement are, often by using the metrics defined in The Three Flow Metrics (Plus One).

The three flow metrics (plus one)

Some of the best indicators of team performance are the flow of both new information into the team and of value out of the team. If we can improve visibility into these indicators, and therefore the opportunities for the team to improve the way they work, it becomes possible for the team to support their organization in ways they couldn’t before. There are three standard metrics that are core to understanding the effectiveness of any flow-based system. The relationship between the three metrics is defined by Little’s Law. When applied to the systems used to enable knowledge work the law is usually restated in terms of Throughput, Work In Progress (WIP), and Cycle Time.

Back to Top ↑

predictability

Slicing stories

In an agile environment, we split our work down into what we call “stories”, that are the smallest unit of value passing through a workstream. Unfortunately, we have a tendency to over-complicate story writing, making it unnecessarily hard. Done well, it can be a simple process of taking small steps, repeatedly.

Keeping people busy

The Kanban Guide talks about optimizing the workflow for three different attributes: effectiveness, efficiency, and predictability. It talks about the fact that any optimizations we perform will be a balance across these three and that over-optimizing on one may make the others worse.

Steps to improving predictability

If you have a need to know when the work will be done or how much you can do in a certain period of time then predictability will be important to you. We have great tools like Monte Carlo for probabilistic forecasting but the truth is that the forecast we generate is only as good as the data we give it. Garbage in yields garbage out. So how do we improve our data to make it inherently more predictable?

One Thing vs Multiple Things

When creating a forecast first ask yourself whether you are forecasting One Thing or Multiple Things. It’s not always clear which of these situations you are in but the approach you take to creating the forecast will differ significantly. This post will help you to figure out which approach to take.

Improving Predictability - Average Age

In a previous post I’ve introduced the four assumptions behind Little’s Law and discussed the first two assumptions in detail. If you haven’t read those previous posts I encourage you to go back to understand the background. As a reminder, the four assumptions are listed below.

Improving Predictability - All work must finish

In a previous post I introduced the four assumptions behind Little’s Law and the idea that they are critical to understanding and improving your system’s predictability. We’ve also already discussed the first assumption regarding the equality of average arrival and departure rates. If you haven’t read those previous posts I encourage you to go back to understand the background. As a reminder, the four assumptions are listed below.

Improving Predictability - Average Arrival and Departure Rates

In a previous post I introduced the four assumptions behind Little’s Law and the idea that they are critical to understanding and improving your system’s predictability. If you haven’t read that post I encourage you to go back to understand the background. As a reminder, the four assumptions are listed below.

Improving Predictability

Little’s Law is an equation that frequently appears in discussions of Kanban systems. While initially formulated as a part of queuing theory to describe the length of time people would spend in stores it has since been applied to many other contexts from manufacturing to knowledge work (particularly Kanban for the purposes of today’s conversation).

Back to Top ↑

waste

Continuous improvement

Back in the days when faxing between companies was a popular thing, I recall a client that had a workflow like this:

  1. Fax arrives and is printed by the fax machine
  2. Paper is picked up by a person and carried to the scanner where is it digitized.
  3. Paper is immediately shredded because there was confidential information on it.

Improving meetings

When we look for opportunities for improvement, at some point every team will bring up meetings as being an ongoing problem for them. When we dig into what the actual problems are, we find they always fall into one of three categories:

Per-story estimates

Per-story estimates were an interesting experiment that failed and it’s time to move on. Today, we have better ways so it’s time to stop putting individual estimates on stories. This is equally true for Scrum and Kanban teams.

Technical Debt

The term “technical debt” is widely used in the industry even if there isn’t a clear definition of it and almost nobody uses the term in the way Ward Cunningham meant when he first coined it. It’s most commonly used to describe things in our environment, usually but not always code, that slow us down. These are things that are working - not bugs - but that are implemented in a poor way that makes them more difficult to understand or modify.

The cost of interruptions and how to reduce it

Interruptions can be a significant source of waste. By their nature, interruptions cause a context switch as we lose track of what we had been working on to focus on the interruption. There is a significant cost to that context switch as it takes time and effort away from the task at hand. There is also a real impact on quality as mistakes are far more likely to happen when we’re distracted.

Waste: Psychological Distress

One of the more subtle forms of waste is psychological distress. When we are afraid or anxious, our sympathetic nervous system activates to prepare us for one of fight, flight or freeze. All good responses in a survival situation.

Understanding waste in the system

In Kanban, we are always trying to optimize for efficiency, effectiveness and predictability. Waste in the system is something that hurts all three of these objectives and is something we want to remove or reduce wherever possible.

Back to Top ↑

probabilistic_forecasting

Monte Carlo under the covers

Monte Carlo forecasting is the most common form of probabilistic forecasting that we see. It’s compelling because it can provide a highly accurate forecast of when work will be done, with relatively little effort.

What is Probabilistic Forecasting?

Do your customers ever ask “When will it be done?” When dealing with the future, there’s almost never an accurate deterministic answer (Tuesday, exactly at 3:45pm) to that question but there is an accurate probabilistic answer (85% chance of completion on or before October 1) and in most cases, it’s a lot easier to calculate than you’d expect.

Forecasting projects

Weather predictions are probabilistic, not deterministic. That means there isn’t a single right answer that we can calculate. We can’t say it will rain at 11:05 but we can say that there’s an 80% chance of rain today. Forecasting when we’ll be done is also probabilistic, in exactly the same way. We can say based on past throughput data that we have an 85% chance of being done on or before May 12.

Per-story estimates

Per-story estimates were an interesting experiment that failed and it’s time to move on. Today, we have better ways so it’s time to stop putting individual estimates on stories. This is equally true for Scrum and Kanban teams.

What is a Service Level Expectation?

A service level expectation is a probabilistic forecast of how long it will take a single item to pass through the system. For example: “85% chance of completing in four days or less”.

Steps to improving predictability

If you have a need to know when the work will be done or how much you can do in a certain period of time then predictability will be important to you. We have great tools like Monte Carlo for probabilistic forecasting but the truth is that the forecast we generate is only as good as the data we give it. Garbage in yields garbage out. So how do we improve our data to make it inherently more predictable?

One Thing vs Multiple Things

When creating a forecast first ask yourself whether you are forecasting One Thing or Multiple Things. It’s not always clear which of these situations you are in but the approach you take to creating the forecast will differ significantly. This post will help you to figure out which approach to take.

Back to Top ↑

improvement

Premature optimization

We have a tendency to think that making any one part of the workflow more efficient will make the overall workflow also more efficient and that’s just not true. Part of that is that not all parts of the workflow are on the critical path and improving something that isn’t currently a bottleneck won’t help the overall flow. But there’s a second reason that’s less obvious - sometimes optimizing for simple cases in the workflow, can make other parts of the workflow much worse.

“We tried Kanban and it didn’t work”

I sometimes run across teams that say “we tried kanban and it didn’t work”. When I hear this, I’m always genuinely curious and ask for more details about what they’d done and what specifically didn’t work for them.

Improve the work, not the metrics

One of the key practices supporting continuous improvement is making your work, and how you do the work, visible. This starts by tracking the progress of that work in a highly visual way, often by using a kanban board. Once that work is being tracked we can begin to gather that data and start to gain insights into where our biggest opportunities for improvement are, often by using the metrics defined in The Three Flow Metrics (Plus One).

Back to Top ↑

stories

Slicing epics

We talk a lot about slicing stories but then when it comes to slicing larger types (epics, features, etc), we tend to wave our hands and say “it’s the same, only bigger”, which while true, is rarely helpful.

Slicing stories

In an agile environment, we split our work down into what we call “stories”, that are the smallest unit of value passing through a workstream. Unfortunately, we have a tendency to over-complicate story writing, making it unnecessarily hard. Done well, it can be a simple process of taking small steps, repeatedly.

Per-story estimates

Per-story estimates were an interesting experiment that failed and it’s time to move on. Today, we have better ways so it’s time to stop putting individual estimates on stories. This is equally true for Scrum and Kanban teams.

Back to Top ↑

cycletime

One Thing vs Multiple Things

When creating a forecast first ask yourself whether you are forecasting One Thing or Multiple Things. It’s not always clear which of these situations you are in but the approach you take to creating the forecast will differ significantly. This post will help you to figure out which approach to take.

Determining cycle time from an online system

This post is aimed at a fairly niche audience so if you aren’t trying to make sense of poor data out of some ticketing system then you might want to skip this one.

Back to Top ↑

reference_class_forecasting

What is Probabilistic Forecasting?

Do your customers ever ask “When will it be done?” When dealing with the future, there’s almost never an accurate deterministic answer (Tuesday, exactly at 3:45pm) to that question but there is an accurate probabilistic answer (85% chance of completion on or before October 1) and in most cases, it’s a lot easier to calculate than you’d expect.

Forecasting projects

Weather predictions are probabilistic, not deterministic. That means there isn’t a single right answer that we can calculate. We can’t say it will rain at 11:05 but we can say that there’s an 80% chance of rain today. Forecasting when we’ll be done is also probabilistic, in exactly the same way. We can say based on past throughput data that we have an 85% chance of being done on or before May 12.

Back to Top ↑

service_level_expectation

What is Probabilistic Forecasting?

Do your customers ever ask “When will it be done?” When dealing with the future, there’s almost never an accurate deterministic answer (Tuesday, exactly at 3:45pm) to that question but there is an accurate probabilistic answer (85% chance of completion on or before October 1) and in most cases, it’s a lot easier to calculate than you’d expect.

What is a Service Level Expectation?

A service level expectation is a probabilistic forecast of how long it will take a single item to pass through the system. For example: “85% chance of completing in four days or less”.

Back to Top ↑

wip

High WIP invalidates prioritization

The more items we have in progress at once (WIP), the less important our initial prioritization is. When we work on one item at a time then items get done in the order we started them so we are completing them in the order of most importance.

Massively overburdened with WIP

About once a year I run across a team that has at least ten times as many items on the board as there are people on the team. The worst I’ve ever seen was a team of ten people with 227 items in progress.

Back to Top ↑

ContinuousImprovement

Continuous improvement

Back in the days when faxing between companies was a popular thing, I recall a client that had a workflow like this:

  1. Fax arrives and is printed by the fax machine
  2. Paper is picked up by a person and carried to the scanner where is it digitized.
  3. Paper is immediately shredded because there was confidential information on it.
Back to Top ↑

WIP

Back to Top ↑

monte_carlo

What is Probabilistic Forecasting?

Do your customers ever ask “When will it be done?” When dealing with the future, there’s almost never an accurate deterministic answer (Tuesday, exactly at 3:45pm) to that question but there is an accurate probabilistic answer (85% chance of completion on or before October 1) and in most cases, it’s a lot easier to calculate than you’d expect.

Back to Top ↑

retrospective

Excessive sub-tasks

I worked with a team once that had gotten into the habit of creating sub-tasks for every little thing they could think of. This list included such obvious items as “write the code” and “test the code” and a typical story would have about twenty of these sub-tasks attached.

Back to Top ↑

standup

Back to Top ↑

throughput

One Thing vs Multiple Things

When creating a forecast first ask yourself whether you are forecasting One Thing or Multiple Things. It’s not always clear which of these situations you are in but the approach you take to creating the forecast will differ significantly. This post will help you to figure out which approach to take.

Back to Top ↑

visualize

Back to Top ↑