Estimating Fixed Price Projects in Open Source Teams

Some time ago, I have demonstrated that it is well possible to plan fixed price projects without direct time estimates despite the fact that the team velocity is permanently changing. Now I was wondering if the method could be applied to open source projects too. Unlike traditional software teams, open source teams are permanently changing and they usually don’t estimate effort estimations but nonetheless produce world class products.

So I picked some projects from the Apache Foundation, namely some of those who use Jira so I could easily extract their work history and do some analysis on it. The first challenge was to extract the velocity from the given data as the teams didn’t estimate in story points. So I looked at the issue types used in those projects and their cycle time histograms. Here is an example from the Apache Zeppelin Project:ZEPPELIN_category_cycle_time.pngI only considered issue types that were not part of other issues so the only ones left were “Bug”, “New Feature”, “Improvement” and “Wish”. From the cycle time histogram, we can see that the average cycle time of Wishes and New Features are roughly two times the average cycle times of Bugs and Improvements. So I assigned each Wish and New Feature two story points and one for each Bug and Improvement. After introducing 14 day sprints I got the following velocity distribution and average cycle time per sprint:

ZEPPELIN_velocity_cycle_time_history.png

A fictional project

In order to do some estimation and check its accuracy, I picked an arbitrary sprint in the middle of the graph and considered only the velocity up to this point (in our case Sprint 1 to 40) for further observation. The result was the following velocity histogram:

ZEPPELIN_prior_velocity

Now I chose to estimate a 200 story point project (bug fixes included). How long would it take the team to finish?

The maximum velocity the team had ever reached was 45 story points so the minimum number of sprints to complete a 200 story points projects would obviously be 5 sprints but with a probability of only 2.5%. The average velocity was 17.93 story points, hence the average number of sprints to complete the project would be 12 sprints. The 95% case requires a Monte Carlo simulation. After 5000 simulated projects, I got the following distribution:

ZEPPELIN_simulation_result_distribution.png

So if we would have committed to finishing the project within 16 sprints, we would be successful in 95% of all cases. Now lets see what really happened after sprint 40 (the blue line in the graph below):

ZEPPELIN_velocity_total.pngAs the team continued to increase velocity, they managed to finish the fictional project in Sprint 48. It only took them 8 sprints to complete 200 story points which was even better than the average of 12 sprints and twice as good as the pessimistic 95% estimate!

Estimating the profit is less straightforward in open source projects than with stable teams (teams that don’t add or remove developers during the project) as the fictional cost per sprint changes with the number of contributors. We’ll have a look at this in the next post.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s