Planning with uncertainty, Part 2

in the first part of this article we have covered the idea behind story points and mentioned the fact that for a team in the real world, the velocity is permanently fluctuating. This makes planning seem difficult but we can easily handle this as I’ll show in the following example.

Let’s say we need to prepare an offer for project. The backlog items have been estimated by the team to be 50 story points. How much do we have to charge the customer to be profitable?

First, we look at the historic data and plot the histogram of the team velocity:
TeamVelocityHistogram
It shows us that the minimum velocity is 1 and that happened once in 24 sprints. If we assume that the velocity distribution will roughly stay the same in the future (yes, there will be uncertainty but we also had plenty of uncertainty in the previous sprints), we can deduce the best case and worst case scenario.

The number of sprints we need in the best case is the number of points in the project (in this case 50) divided by the maximum velocity which is 25 which gives us 2 sprints. The probability that this happens is the probability of having a velocity of 25 to the power 2 because we are done in 2 sprints.

We can do the same thing with the worst velocity (1 in our histogram), calculate the number of sprints we need when the velocity is one which gives us 50 sprints. As we have to multiply a low probability 50 times with itself, we get an extremely low total probability of this case of about ten to the minus 50. This is actually good news because it tells us that we can safely neglect this scenario.

Finally, we calculate the average number of sprints required by taking again the story points of the project divided by the mean velocity.

Here is a summary of the calculations we did:

VelocityEdgeCases

so we know what we can expect in the best or worst case. This however doesn’t tell us really something about the future. There is always chance that all team members get sick and the velocity will drop to zero. But we have seen that the calculations made from historic data cover a wide range of scenarios.

Getting a more detailed picture using traditional probability theory and combinatorics is hard so we use a very simple and straight forward  approximation called the Monte Carlo method. Instead of stressing combinatorics to reason about all possible outcomes of a project, we use a random number generator to produce velocities with the distribution provided by the histogram we made from historic data.  This way, we can simulate a large number of projects and analyse the outcomes of those projects.

In each project we add up randomly generated story points until we reach 50, the number of story points we need to complete the project. Then, we record the number of sprints we needed to finish each project. The result from 100 simulated projects looks something like this:

SimulatedProjects
As we can see, most projects finish between 3 and 4 sprints. There are some projects where the team could really do it in 2 sprints and some rare projects with a velocity of 5 and 6. Next, we have a look at the histogram of the project duration after having simulated 5000 projects:
ProjectDurationHistogram
This confirms our observation from before. From this we can figure out how many sprints we need to finish the project in at least 95% of all cases – this is usually paranoid enough. This observation requires the presence of some rare cases (7 sprints and more) in our data, thus the need for the large number of simulated projects.

ProjectDurationHistogramData

The number of sprints required for projects finishing in at least 95% of all cases can be directly seen in the histogram data. It’s the probability of 1-0.95 which is 0.05 which is reached with 6 sprints.

We start making money at average when the probability of succeeding exceeds 50%, so we look at the histogram data again and spot it between 2 and 3. It’s actually 2.64, the value deduced from the mean velocity we calculated before. If we sell the customer 6 sprints, we’ll make a profit in 95% of projects and gain 3.36 sprints at average.

SprintsGain

This shows that the uncertainty expressed by the fluctuation of the velocity doesn’t prevent us from making money. It’s the average velocity, not the predictability that brings the money.

I have used Jupyter for the calculation, the graphs and formulas. It saved me a lot of time and struggle compared to Excel. The Notebook can be found here.

2 thoughts on “Planning with uncertainty, Part 2

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s