Estimating for the bottleneck

Your Agile team may employ an estimation technique that requires the participation of all team members. But this kind of all-hands involvement is completely unnecessary if you know where your bottleneck is.

Finding the bottleneck

If you can picture your development process as a chain of dependent operations that are performed in sequence (e.g. development, testing, deployment), then one of those operations will be your bottleneck. And just so we’re clear about what is meant by “bottleneck”, here’s a working definition:

A bottleneck is any resource with no spare capacity.

In other words, one of the operations in your development chain is going to set the pace—to determine the rate at which work items are completed. Find the operation whose throughput matches the throughput of the entire chain, and you’ve pegged your bottleneck.

All-hands estimation is unnecessary

Conventional estimation sessions, even within the Agile community, include input from all team members. It’s common to see teams arriving at an estimate for a given unit of work (e.g. user story, backlog item) by summing the anticipated efforts of each operation. (Development will take 3 days and testing will take 2 days, therefore the final estimate is 5 days.)

The problem with the above scenario is that it assumes that no work is done in parallel, which is rarely the case. In reality, development and testing will be occurring simultaneously most of the time, and the production rate of the bottleneck operation will determine the production rate of the entire chain.

When all operations in the chain are working in parallel, and assuming that there are many units of work to be done, then the time to complete a given unit of work is determined solely by the time it spends at the bottleneck operation. Therefore, in the above scenario, the time to complete the unit of work will be closer to 3 days than to 5.

This means that there’s really no need to involve the entire team to arrive at estimates. Getting estimates from team members involved in the bottleneck operation only will yield appropriate results.

What if we can’t find our bottleneck?

I often hear teams reporting that they have multiple bottlenecks, or that the bottleneck jumps from operation to operation, making it difficult to “estimate for the bottleneck.” If this appears to be the case in your development chain, then you’re most likely observing BINOs (Bottlenecks In Name Only). In any give chain of dependent operations, there will be one true bottleneck. There are several techniques for revealing where it is:

  • Observing Inventory Buildup – Find the operation that has the largest growing volume of work to be done.
  • Matching Cycle Times – Find the operation whose cycle time matches the cycle time of the entire chain.
  • Assignment – Where two or more operations have similar production rates, choose the one you wish to set the pace for the entire chain.


Although the example above talks about estimating in absolute time (days), I highly recommend estimating in relative time (points). We’re generally not very good at estimating in absolute terms, but can be quite adept at relative estimation (e.g. unit of work A will take about twice the time as unit of work B).

See my previous article, What are we estimating, anyway?

Read more about estimation in the 99-cent Kindle pocket guide, Practical Estimation.

That bottleneck may be right where you want it

Mere mention of the word “bottleneck” is often enough to generate a flurry of proposals on ways to eliminate it. But every system has a bottleneck, and yours may be right where you want it.

A useful definition

Examples of bottlenecks, both at work and out in the world, abound. Work piles up on the desk of a co-worker who is out sick. Traffic comes to a standstill on the freeway while driving to and from work. (The most common one in my commute is a particular spot on Highway 101 north of Novato where three lanes turn into two.)

But some things that we think of as bottlenecks, aren’t. In a work environment, it’s common for people to label anyone or anything that is preventing them from starting their own next work task as a bottleneck.

A more useful definition of bottleneck comes from The Goal by Eliyahu M. Goldratt:

“A bottleneck is any resource whose capacity is equal to or less than the demand placed on it.”

That’s a powerful statement. And it’s the “equal to” part that I find the most interesting. If you turn the definition on its head, you can express it as:

A bottleneck is any resource with no spare capacity.

The bottleneck is a pacesetter

It’s convenient to think of a software development environment as a chain of dependent operations (e.g. requirements gathering, design, development, testing, deployment).

The bottleneck in your software development environment is the operation with no spare capacity with respect to the demand on your chain of development operations as a whole. If you measure the rate at which finished product is leaving the end of the chain, you can identify the bottleneck by finding the operation whose production rate matches the system’s rate.

In other words, the bottleneck operation sets the pace of production for the entire system. And that’s why I find it useful to refer to a true bottleneck as a pacesetter.

Deciding whether or not to eliminate a bottleneck

Your system is going to have a bottleneck somewhere, and fixing a bottleneck at one operation will have the effect of moving it to some other operation. You only want to move a bottleneck if you don’t like it where it is.

So how do you know if your bottleneck is in the right place? Just take a look at the rate of production of your development environment as a whole. If delivery of finished product (e.g. features, user stories, or whatever units of work you use) is keeping up with demand, you’ll likely want to leave the bottleneck where it is.

And how do you assess demand? In a software development environment, demand might be determined by your marketing organization (i.e. “We need to have features A, B and C delivered by the end of March”).

A bottleneck side-effect

A development environment that is keeping up with demand, and that has a well-managed bottleneck, will also exhibit what is often perceived as an undesirable trait: Not every operation in the development chain will be occupied 100% of the time. In other words, some people are going to appear as if they’re not working at least part of the time.

But this is intentional. If the bottleneck is the pacesetter for the entire system, then all of the non-bottlenecks will, by definition, have spare capacity which can be used, when appropriate, to make sure that the pacesetter operation is never starved for work.



Grow your own best practices

Development organizations are often concerned with finding out what “best practices” they should employ to help ensure a successful product. But practices don’t always transplant well. Rather than adopting someone else’s practices, you might want to derive your own from basic principles.

The bowling story

Many years ago I took my younger daughter to the bowling alley for an afternoon of fun. I hadn’t been to this particular bowling alley before, so was unfamiliar with their policy regarding shoe rental.

When I asked the young lady behind the desk for a lane and shoes, she responded, rapid fire with, “I-need-a-shoe-size-and-a-shoe.” It sounded like all one word.

I didn’t understand what she had said, so I asked her to repeat it. Again she said, “I-need-a-shoe-size-and-a-shoe.” This time I caught the words “shoe size,” so I replied, “Ten and a half.”

She plopped a pair of size ten-and-a half bowling shoes on the counter, then said, “And a shoe.”

I was puzzled. “What?” I asked her.

She repeated, “And a shoe.”

“I don’t know what you mean,” I replied.

“I need one of your shoes.”

“Why?” I asked.

Instead of answering me, she pointed to a sign on the front of the counter which said something to the effect that customers are required to leave one of their shoes at the front desk at the time of bowling shoe rental.

I looked back at the young lady (who was probably already tired of dealing with me) and said, “That’s not why, that’s what.”

A few moments of awkward silence told me that she was disinclined to explain further, so I gave her one of my street shoes, collected the bowling shoes, and walked toward the lane she had assigned us, a little off-balance, one shoe off and one shoe on. (Diddle diddle dumpling, my son John…)

Bowling alley policy – principle vs practice

By pointing to the policy statement, the young lady was indicating the bowling alley’s practice regarding shoe rental. That told me the what, but not the why. By the time I arrived at my assigned lane, I was able to infer the why, which likely had something to do with people literally walking off with bowling shoes. (A quick conference with my daughter tended to confirm this hypothesis. She told me that a number of her friends occasionally wore bowling shoes to school.)

Thus I was able to infer the principle involved, which might be expressed as “some customers will steal our property unless we hold onto something of theirs that they value more than our property.”

Practices may not transplant well

Now let’s suppose that the owner of another business, say a miniature golf course, decided to enact a similar policy (after, perhaps, speaking to the bowling alley owner who expressed that theft was all but eliminated after implementing the “shoe policy.”)

And let’s further suppose that the miniature golf owner hopefully and enthusiastically has a sign made up and displayed prominently, stating that all people playing miniature golf must leave one of their shoes at the counter, hoping to eliminate theft of the occasional putter.

You can imagine the results. Rather than reducing theft, this policy will likely have the effect of reducing revenue, by thinning the golf course’s customer base down to those who enjoy playing minus a shoe.

And, of course, no reasonable person would actually implement a policy that was as ill-fitting as this. The mismatch is too obvious. Instead the owner of the miniature golf course is likely to infer a principle from the bowling alley practice, then implement a similar practice appropriate to the miniature golf business, perhaps something like all people playing miniature golf must leave a small cash deposit at the counter.

Principle vs practice in a development environment

Development (and especially software development) organizations seem to like the idea of finding and applying what they call best practices. I’m not aware of any common agreement on what “best” means, but in my experience it almost always involves “something that someone else is doing.”

And while the differences between software development organizations may not be as glaringly obvious as those between a bowling alley and a miniature golf course, suffice it to say that differences (e.g. size, experience level, customer type, product type, etc.) do exist.

Pair programming is one example of a development practice that’s been around for a while now. In pair programming, as the name suggests, two developers work side by side, on the same code at a single computer. Proponents of pair programming cite increased quality, better design and instantaneous code review as some of its benefits.

I’ve worked on several projects where attempts to implement pair programming were made. Some worked well. Others, not so much. Generally speaking, I’d say that the unsuccessful cases occurred due to social, rather than technical, reasons. In all instances, everyone attempting the practice was technically skilled. But personality or style mismatches sometimes got in the way, making pair programming something other than a best practice in those cases.

Looking at principles rather than practices might have been helpful. So what principles might we have been able to infer? These come to mind:

  • Having more than one person review code will decrease the defect rate.
  • Code that is understandable to more than one person is more maintainable, and therefore less costly to the organization.
  • Simpler designs result from collaboration

From the principles listed above, can you derive some practices that fit the unique composition and characteristics of your organization? I’m guessing that you can. What are they? I can’t say—I’m not in your organization. The practices that you come up with are based on generally applicable principles, but the specifics of those practices are highly dependent on your particular work environment. (In other words, you’ll have to figure them out for yourself.)

I’m interested to hear what you come up with. Are there specific practices you can derive from the principles listed above that would serve you better than pair programming? Please feel free to comment.


What are we estimating, anyway?

Several schools of thought exist regarding the proper units to use when estimating work effort, especially on Agile teams. This article addresses what just may be “the simplest units that could possibly work” when estimating in order to provide predictability of development schedule.

When I first encountered what was called “Agile estimating” in 2004 or so, it was common to hear people talk about estimating user stories in terms of “size.” The emphasis was not on how long a user story would take, but rather on how big a story was. This seemed like a reasonable viewpoint, as it tended to de-emphasize duration, something that experience told me I’m not very good at assessing. (I tended to be overly optimistic.)

But as I attempted to put this emphasis on size into practice, I discovered that it was difficult to explain what “size” meant. I thought I had a grasp of it myself, but was generally unable to articulate what I meant by “size” to others on my team, and with whom I needed to engage in estimation activities. Often the best I could do was to define size in terms of what is was not (e.g. “it’s not duration” or “it’s not time, it’s size“).

Not satisfied with the inability to communicate clearly about this, I began thinking of and explaining “size” in terms of relative complexity. This was somewhat more satisfying, as the concept was easier to communicate, and my teammates and I seemed to be able to talk about this and use it. We decided that “less complexity” could be equated to “having fewer parts” and that “more complexity” meant “having more parts.”

This was fine as far as it went, but there were still some issues. It sometimes turned out that stories of supposedly equal complexity took substantially different amounts of time to complete, making the predicting of schedules less reliable. Order of story development played a part here. When two equally complex stories shared some underlying infrastructure, the first story to be developed would take longer than the second.

Some time later, I learned of other teams estimating in terms of what they called “effort.” This had a certain appeal, but seemed to raise old issues. Coming to agreement on what was meant by “effort” felt about the same as agreeing about the meaning of “size.” Effort is a slippery term, as it can refer to not only how long something will take, but also the ease with which that thing is accomplished. In other words, a user story that is easy (i.e. enjoyable) to work on may seem almost effortless, yet take more time to complete than a story that is difficult (i.e boring).

It could very well be that the appeal in using any of the above units, (size, complexity or effort) lies in our desire to avoid talking about estimates in terms of time. Perhaps we’ve convinced ourselves that we shouldn’t be talking about time, but about something (anything) else when estimating work effort.

But we should also remember why we’re estimating in the first place. If we’re estimating to provide predictability of schedule, then time is what we’re interested in. I’ll go so far as to state that while we’re not generally very good at estimating in terms absolute time, we’re much more adept at estimating in terms of relative time. (e.g. “I don’t know how long stories A and B will take to develop, but I do know that story A will take about twice as long as story B.”)

Given a choice of all the units described above, relative time may be “the simplest units that could possibly work.” They’re certainly more closely related to what we really want to know (absolute time), and they avoid the ambiguities that seem to be inherent in the other units.

Read more about estimation in the 99-cent Kindle pocket guide, Practical Estimation.

Getting started

I’ve been learning about Agile, and employing its principles and practices, for a while now.

I was introduced to some of the tenets of eXtreme Programming (XP) in 2000 by a co-worker at a software development company. I liked what I saw, but didn’t get a chance to apply it much at the time. My next opportunity came in 2005, when I got involved in another software development effort that started out in the traditional “waterfall” fashion with a good-sized design document, but changed over to an Agile approach because we all knew that the requirements for the product being built were going to be changing constantly.

My personal discovery of Agile principles and practices renewed my interest in software development (in which I had been involved since 1977). Agile held a promise that I could actually be part of creating some fairly large software systems without them becoming inflexible and unmanageable with growth.

As I learned more, I began to realize something that many others almost certainly know: This Agile stuff has much broader application than just in the world of software. So I adopted a broader viewpoint, and began thinking in terms applying Agile principles and practices to productivity in general.

I’m a big fan of Eliyahu Goldratt’s Theory of Constraints, and find myself reading and re-reading his body of work. I like Goldratt’s clear and concise definition of productivity, and use it myself in practically everything I do work-wise.

Eventually I began training and coaching other people in Agile principles and practices, and developing hands-on exercises that help get the basic ideas across. I’m still doing that today, and intend to keep on doing it, hoping to reach larger and larger audiences.

I’m also hoping that this blog will serve to promote the exchange of ideas between myself an other enthusiasts out there. Please feel free to join the conversation.