To estimate, or not to estimate?

That is the question…

The practice of estimating how long it will take to implement work items in order to predict project schedules is being replaced, in some organizations, by a different practice: counting the number of work items completed in a given time period. This article addresses both approaches in an attempt to help you determine which is right for your situation.

Traditional Estimation

You are likely familiar with the traditional way that estimates are created:

  • Learn all you can about the requirements of the product.
  • Create a list of all the activities you can think of that need to be performed.
  • Assign a time (typically in hours) to each task indicating how long you think it will take to do.
  • Add up all those numbers.
  • Add some padding (perhaps 50 to 100%) to the total.
  • Resist your manager’s attempts to shave that number down.
  • Fight the impulse to quit your job when you see that your estimate has become part of the official project schedule.

Anyone who has employed this approach to estimation knows that it can often be time-consuming and unproductive. Fortunately, there are alternatives.

Relative Estimation

The Agile software development philosophy brought with it the theory that people are naturally better at estimating in relative terms (e.g. “which of those two buildings is taller?”) than they are at estimating in absolute terms (e.g. “how tall is that building?”).

Some Agile teams use Fibonacci numbers (1, 2, 3, 5, 8, 13…) for relative estimation of work items because the numbers grow at a reasonable rate and because they express the fact that estimates, being educated guesses, are not precise. The numbers typically represent abstract units called points.

At the end of each work period (e.g. iteration, sprint) team members sum the estimates associated with each completed work item to yield the team’s velocity, commonly expressed in terms of points per iteration. They then use this velocity figure to forecast future schedules.

Item Counting

Some teams don’t estimate at all, even in relative terms, preferring instead to simply count the number of work items completed in a given time period, and use that number to forecast future schedules.

As you might imagine, more accurate forecasts are to be had when the work items are small and close to the same size (meaning that they take about the same amount of time to implement).

This implies that some work must be done in order to break larger work items down into smaller work items, each of similar duration.

Estimation vs Item Counting

Both estimation and item counting can be used to forecast future completion dates. Each method has its own characteristics which can be considered when you are deciding whether or not to use it.

Consideration Estimation Item Counting
Overhead It takes time to go through work items and establish an estimate for each one. It takes time to go through work items and break larger items into smaller ones of similar size.
Scope Estimates can be made for all work items in a product, regardless of item size. This makes it possible to forecast project completion after just one iteration of work without breaking down all items into smaller ones. Large work items must be broken down into smaller items before they can be implemented and subsequently counted. Project completion can be forecast only after all items have been broken down.
Forecasting Schedule forecasting is accomplished by projecting the number of points that can be completed in the future, based on historical velocity. Schedule forecasting is accomplished by projecting the number of work items that can be completed in the future, based on historical work item count.
Availability A dependable forecast cannot be made until some work has been done. A dependable forecast cannot be made until some work has been done.

How much overhead is acceptable?

Both estimation and item counting have associated overhead that is incurred before scheduling forecasting can be performed. When estimating, the overhead is the time the estimation itself takes. When counting work items, the overhead is the time it takes to break larger work items into small ones, all of a similar size (duration).

How much overhead would you tolerate in order to be able to predict future completion dates to within an acceptable margin of error? Four hours per week? Two hours? One? Everyone’s situation is different, but I imagine most would agree that the less overhead incurred, the better.

Deciding which method to use

Like many other things in the Agile world, there’s a simple way to decide whether or not to use any particular practice: Try it! Yes, that’s right–as in most other Agile practices, the decision to use one of these methods or the other needs to be based on empirical evidence. In other words, you have to actually do some work in order to get answers.

More specifically, you can perform an experiment to see which method works best for you. It might go something like this:

Experiment setup

  • Chose a relatively small project, one that you think will produce a completed product in about six weeks. Assume 1-week iterations.
  • Create a set of work items (e.g. user stories) covering the entire product. A mix of large and small items is best for this experiment. (“Large” in this case means “too big to implement in one iteration.”)

Experimenting with relative estimation

  • Perform relative estimation on all work items. Record how long the estimation took.
  • You need to have from 5 to 10 small work items for the first iteration. If you don’t, break some larger items into small ones until you do. Record how long it took to create the smaller items.
  • Attempt to implement as many items as you can in the first iteration (week).
  • Count the number of points actually completed. That’s your velocity.
  • Forecast a project completion date based on that velocity.
  • If you don’t have from 5 to 10 work items for the second iteration, break some larger items into small ones until you do. Record how long it took to create the smaller items.
  • Attempt to implement as many items as you can in the second iteration.
  • Forecast a new project completion date based on a new velocity figure.

 Experimenting with item counting

  • Break all work items, large and small, into items all similar in size, and small enough so that each iteration will contain 5 to 10 items. Record how long it took to create the smaller work items.
  • Attempt to implement as many items as you can in the third iteration.
  • Count the number of items actually completed. That’s your item count.
  • Forecast a project completion date based on that item count.
  • Attempt to implement as many items as you can in the fourth iteration.
  • Forecast a new project completion date based on the new item count.

Evaluating the results

  • Compare the combined estimation and work item breakdown times in the first half of the experiment to the work item breakdown time in the second half of the experiment. Is one substantially less than the other?
  • Compare the number of points competed in the first iteration to the points completed in the second iteration. Are they reasonably close to each other?
  • Compare the number of work items competed in the third iteration to the items completed in the fourth iteration. Are they reasonably close to each other?
  • Compare the project completion dates from the first half of the experiment to the completion dates from the second half of the experiment. Does one pair seem more reasonable than the other?

The answers to the above questions may be enough to help you decide whether you want to use relative estimation, work item counting (or both) in your product development work. Some teams start with relative estimation, then move on to work item counting as they gain experience with making little work items out of big ones. I suggest you try them both to see which works best for you.

Read more about estimation in the 99-cent Kindle pocket guide, Practical Estimation.

Dude, where’s my user?

When crafting user stories, is it OK to name some non-human part of the system as your user? You may be creating requirements masquerading as user stories.

A common user story format

You are probably already familiar with this standard user story format:

As a <type of user>
I want <some capability>
So that <some benefit>

During the course of working with various development teams, I’ve run across stories like this more often than I would have imagined:

As the database, I want to have a Customer table, so that there is a place to store customer information.

or:

As the system, I want to have an account object, so that account information can be seen.

The examples above certainly follow the standard format for user stories, but that doesn’t necessarily mean we’re getting all that we can from them. Either could be re-written as a simple requirement:

The database shall store customer information in a Customer table.
The system shall store account information in an Account object.

What we have here is a case of requirements masquerading as user stories. Note that both the faux and true user stories specify what is to be built. The difference is that the faux user stories leave out important information that true user stories convey, namely who the user is and why the story is being implemented.

These are important pieces of information, as they aid in determining that we’re building the right thing. In short, if a developer knows why something is to be built, and for whom, then he or she can make more intelligent decisions about precisely what is needed. And this can result in building something that is more useful and less expensive.

So if a simple requirement expresses the same thing as a so-called user story, why go to the trouble of bending the requirement into the shape of a story? Why do we see as many examples of faux user stories as we do?

My guess at the most common reason is that developers have been taught, or have led themselves to believe, that all features of a product must be expressed as user stories.

Another reason may stem from claims I’ve heard fairly often that are typically expressed as something like, “The work I do doesn’t have a user.”

Scrum says nothing about user stories

Scrum is the most commonly used framework for Agile development. And contrary to what many have been taught, Scrum says nothing about work items being expressed as user stories, but instead refers to them generically as Product Backlog items. Additionally, Scrum does not even require that all backlog items be of the same type or format, so a mix can be perfectly acceptable.

Now, to be clear, I’ve got nothing against user stories. I think they’re a very effective way to express the capabilities a system provides, and I use them almost exclusively when doing development work. My objection is to time-wasting efforts to express all work to be done in user story format.

Depending on your development environment and culture, you may be most comfortable populating your Product Backlog with a mix of true user stories and technical tasks.

My personal preference is to put only true user stories in the backlog, and to let any user story spark the creation of any necessary technical, under-the-hood stuff. But to do that, I have to engage in a mental exercise that I call:

Dude, where’s my user?

First, when I say “user” I mean some human-type person—not a data structure, a software component or other aspect of the system being built or a system that uses it.

User doesn’t necessarily mean a person interacting with some user interface. It means the person using your product, whatever it is that you produce. So if, for example, you produce web front-ends, your user probably is interacting with a user interface, and you should have little trouble identifying that user. If, however, you produce an API for a web service, your user is likely to be another developer, and you can write your user stories accordingly.

If you can’t easily identify one, you need to ask yourself, “Dude, where’s my user?” Because there probably is one out there somewhere. You may need to follow the path from your product through one or more other connected systems, until you find that person.

And If you can’t ultimately identify a user who benefits from what you are about to build, perhaps you should reconsider whether you’re building the right thing. Products are ultimately created for the benefit of human beings, so if you can’t find those people, either keep looking or stop building.