Weather predictions are probabilistic, not deterministic. That means there isn’t a single right answer that we can calculate. We can’t say it will rain at 11:05 but we can say that there’s an 80% chance of rain today. Forecasting when we’ll be done is also probabilistic, in exactly the same way. We can say based on past throughput data that we have an 85% chance of being done on or before May 12.
If we have an idea of how many work items we have, then we can use a Monte Carlo simulation to forecast that with significant accuracy. But what about those cases where we’re starting a new initiative or project and have no real understanding of how many work items will be involved?
If we’ve sliced our epics well in the past then we can still use Monte Carlo at the epic level and that may be good enough. The bad news is that most companies do a poor job of slicing epics and so this doesn’t provide as much help as we’d hope.
What we can do instead, is look at historical base rate data for past work.
Base rates are historical examples for similar kinds of work. We’ll almost never have something that’s exactly the same but we will have things that are comparable enough.
There are two different views (inside and outside), that we need to consider to get a reasonable forecast and yet interestingly, most people will only consider one (the inside). If we don’t consider both then our forecast will be considerably less accurate.
Let’s look at a simple example:
An individual has been described by a neighbor as follows: “Steve is very shy and withdrawn, invariably helpful but with little interest in people or in the world of reality. A meek and tidy soul, he has a need for order and structure, and a passion for detail.”
Is Steve more likely to be a librarian or a farmer?1
Inside view: We know a number of attributes about Steve himself and these are all part of the inside view. “shy and withdrawn, invariably helpful but with little interest in people or in the world of reality. A meek and tidy soul, he has a need for order and structure, and a passion for detail.”
This tells us a lot about the uniqueness of Steve but not about the environment that Steve is working within. We need the outside view for that.
Outside view: Statistically we know that there are many more farmers in the overall population than there are librarians. For example, a quick google search shows 30 times more farmers in Canada than there are librarians.
The outside view is statistical and is based in measured data. The inside view is narrative and story driven and is heavily affected by cognitive bias.
To get the most accurate forecast, we need to consider both and yet studies done by Daniel Kahneman1 on this kind of forecasting “found that participants in our experiments ignored the relevant statistical facts and relied exclusively on resemblance.”
In other words, they used the inside view exclusively, significantly reducing the accuracy of their forecast.
For the problem above, if we only consider the inside view, then Steve is clearly more likely to be a librarian. Yet when we include both the inside and outside, Steve is more likely to be a farmer.
Imagine a situation where we are running an IT shop and are being asked to forecast how long a new project will take.
Let’s consider both the inside and outside views, except this time we’ll do outside first as it will set important context.
Note that the questions we list here are common but are not necessarily comprehensive. Use your best judgement to compare the right factors in your case.
Outside view: Here we’ll look across past projects to collect relevant base data. It’s unlikely that any project will be exactly the same so we’ll need to consider what differences there are between each.
For similar past projects
- In what way are those projects different from the others?
- How long had they taken to complete?
- How much had they cost?
- How many people had worked on them? (an adjustment factor for cost)
- What dependencies had those projects had?
- How often have we met the deadline previously?
Inside view: This is where we look at all the unique factors for this particular piece of work.
- Complexity
- Competing priorities. Is this the only thing we’re working on or do we expect our attention to be split?
- Availability of people and resources.
- Anticipated risks and delays. Ie there’s a 30% of being audited while working on this and if we are, it will take 2 months longer.
Combining the two The outside view will give us a range of possibilities and it’s highly likely that this new work will fall somewhere inside that range. The inside view will help us determine where in the range it might fall.
To reiterate, most people will consider only the inside view and will end up with a far less accurate forecast. By considering both, we can get a result that is far more accurate, without a tremendous effort.
We’ve used this technique on new product launches and have discovered that even when people have been told up front about the inside and outside views, they’ll often still ignore the outside and need to be reminded. Our implicit bias towards the inside is very strong.
-
Thinking Fast and Slow by Daniel Kahneman ↩ ↩2