To estimate, or not to estimate?

That is the question…

The practice of estimating how long it will take to implement work items in order to predict project schedules is being replaced, in some organizations, by a different practice: counting the number of work items completed in a given time period. This article addresses both approaches in an attempt to help you determine which is right for your situation.

Traditional Estimation

You are likely familiar with the traditional way that estimates are created:

  • Learn all you can about the requirements of the product.
  • Create a list of all the activities you can think of that need to be performed.
  • Assign a time (typically in hours) to each task indicating how long you think it will take to do.
  • Add up all those numbers.
  • Add some padding (perhaps 50 to 100%) to the total.
  • Resist your manager’s attempts to shave that number down.
  • Fight the impulse to quit your job when you see that your estimate has become part of the official project schedule.

Anyone who has employed this approach to estimation knows that it can often be time-consuming and unproductive. Fortunately, there are alternatives.

Relative Estimation

The Agile software development philosophy brought with it the theory that people are naturally better at estimating in relative terms (e.g. “which of those two buildings is taller?”) than they are at estimating in absolute terms (e.g. “how tall is that building?”).

Some Agile teams use Fibonacci numbers (1, 2, 3, 5, 8, 13…) for relative estimation of work items because the numbers grow at a reasonable rate and because they express the fact that estimates, being educated guesses, are not precise. The numbers typically represent abstract units called points.

At the end of each work period (e.g. iteration, sprint) team members sum the estimates associated with each completed work item to yield the team’s velocity, commonly expressed in terms of points per iteration. They then use this velocity figure to forecast future schedules.

Item Counting

Some teams don’t estimate at all, even in relative terms, preferring instead to simply count the number of work items completed in a given time period, and use that number to forecast future schedules.

As you might imagine, more accurate forecasts are to be had when the work items are small and close to the same size (meaning that they take about the same amount of time to implement).

This implies that some work must be done in order to break larger work items down into smaller work items, each of similar duration.

Estimation vs Item Counting

Both estimation and item counting can be used to forecast future completion dates. Each method has its own characteristics which can be considered when you are deciding whether or not to use it.

Consideration Estimation Item Counting
Overhead It takes time to go through work items and establish an estimate for each one. It takes time to go through work items and break larger items into smaller ones of similar size.
Scope Estimates can be made for all work items in a product, regardless of item size. This makes it possible to forecast project completion after just one iteration of work without breaking down all items into smaller ones. Large work items must be broken down into smaller items before they can be implemented and subsequently counted. Project completion can be forecast only after all items have been broken down.
Forecasting Schedule forecasting is accomplished by projecting the number of points that can be completed in the future, based on historical velocity. Schedule forecasting is accomplished by projecting the number of work items that can be completed in the future, based on historical work item count.
Availability A dependable forecast cannot be made until some work has been done. A dependable forecast cannot be made until some work has been done.

How much overhead is acceptable?

Both estimation and item counting have associated overhead that is incurred before scheduling forecasting can be performed. When estimating, the overhead is the time the estimation itself takes. When counting work items, the overhead is the time it takes to break larger work items into small ones, all of a similar size (duration).

How much overhead would you tolerate in order to be able to predict future completion dates to within an acceptable margin of error? Four hours per week? Two hours? One? Everyone’s situation is different, but I imagine most would agree that the less overhead incurred, the better.

Deciding which method to use

Like many other things in the Agile world, there’s a simple way to decide whether or not to use any particular practice: Try it! Yes, that’s right–as in most other Agile practices, the decision to use one of these methods or the other needs to be based on empirical evidence. In other words, you have to actually do some work in order to get answers.

More specifically, you can perform an experiment to see which method works best for you. It might go something like this:

Experiment setup

  • Chose a relatively small project, one that you think will produce a completed product in about six weeks. Assume 1-week iterations.
  • Create a set of work items (e.g. user stories) covering the entire product. A mix of large and small items is best for this experiment. (“Large” in this case means “too big to implement in one iteration.”)

Experimenting with relative estimation

  • Perform relative estimation on all work items. Record how long the estimation took.
  • You need to have from 5 to 10 small work items for the first iteration. If you don’t, break some larger items into small ones until you do. Record how long it took to create the smaller items.
  • Attempt to implement as many items as you can in the first iteration (week).
  • Count the number of points actually completed. That’s your velocity.
  • Forecast a project completion date based on that velocity.
  • If you don’t have from 5 to 10 work items for the second iteration, break some larger items into small ones until you do. Record how long it took to create the smaller items.
  • Attempt to implement as many items as you can in the second iteration.
  • Forecast a new project completion date based on a new velocity figure.

 Experimenting with item counting

  • Break all work items, large and small, into items all similar in size, and small enough so that each iteration will contain 5 to 10 items. Record how long it took to create the smaller work items.
  • Attempt to implement as many items as you can in the third iteration.
  • Count the number of items actually completed. That’s your item count.
  • Forecast a project completion date based on that item count.
  • Attempt to implement as many items as you can in the fourth iteration.
  • Forecast a new project completion date based on the new item count.

Evaluating the results

  • Compare the combined estimation and work item breakdown times in the first half of the experiment to the work item breakdown time in the second half of the experiment. Is one substantially less than the other?
  • Compare the number of points competed in the first iteration to the points completed in the second iteration. Are they reasonably close to each other?
  • Compare the number of work items competed in the third iteration to the items completed in the fourth iteration. Are they reasonably close to each other?
  • Compare the project completion dates from the first half of the experiment to the completion dates from the second half of the experiment. Does one pair seem more reasonable than the other?

The answers to the above questions may be enough to help you decide whether you want to use relative estimation, work item counting (or both) in your product development work. Some teams start with relative estimation, then move on to work item counting as they gain experience with making little work items out of big ones. I suggest you try them both to see which works best for you.

Read more about estimation in the 99-cent Kindle pocket guide, Practical Estimation.

Leave a Reply