Why estimates are wrong in one direction

The observation

Estimates are not randomly wrong. They are wrong in one direction, consistently, across organisations, domains and decades.

That consistency is the interesting part. Random error would cancel over many projects and it does not, which means the cause is structural rather than a failure of care.

The shape of the distribution

The simplest cause requires no psychology at all. A task cannot finish in less than no time, so the distribution of outcomes is bounded below. Nothing bounds it above: a task can take twice as long, or ten times, or never finish.

An estimator naming the most likely duration therefore names a value below the average, because the long tail on one side pulls the average away from the peak. Even a perfectly calibrated guess about the commonest outcome produces a plan that is late more often than early.

1979Estimating from the inside

The psychological account, from work on prediction generally, is that people estimate by imagining the plan and adding up its parts.

The plan contains only what was thought of. Everything omitted is omitted from the estimate too, and what is omitted is systematically the same category: integration, waiting for other people, the thing that turns out to be harder than it looked, and the work created by discovering that.

The remedy that works

Estimate from the outside instead. Take the class of task, find how long similar ones actually took, and use that distribution rather than reasoning about this one.

It is unpopular because it feels like ignoring what you know about the specific case, and it outperforms the detailed reasoning reliably. The information it uses is the information that the detailed reasoning omits.

The multiplier every team could compute and almost none do

Record the estimate, record the actual, divide. After a few dozen tasks you have your own distribution, which is far more useful than any published figure because it includes your organisation's particular sources of delay.

Very few teams keep this record, and the reason is not difficulty. A recorded multiplier makes optimism visible and attributable, and nobody in the estimating chain benefits from that.

The incentives, said plainly

Optimistic estimates get approved. Pessimistic ones get challenged, reduced, or lose to somebody else's optimistic one.

So the selection pressure is on the estimate rather than on the estimator, and an organisation that punishes overruns without recording estimates is selecting for people who are confident rather than for people who are right.

2010sThe position that gives up on it

One school argues that estimation for most software work is not worth its cost: the effort spent producing numbers that will be wrong would be better spent making the work smaller and shipping continuously, so that the question rarely needs answering.

It is a serious position, and it depends on being able to slice work finely and deliver often, which is available to some kinds of organisation and not to others. Where a commitment must be made to somebody outside, the question returns.

Why splitting the work helps and how it fails

Breaking a task into smaller ones does improve estimates, because the errors on independent pieces partly cancel and because small pieces contain fewer unimagined parts.

It fails when the decomposition itself is the estimate: listing the pieces you can think of is exactly the inside view, performed at higher resolution. The pieces nobody listed are still missing, and there are now more places for them to hide.

The estimate that is really a commitment

A large share of the difficulty is that two different things share one word. A prediction is a statement about the world and can be wrong without anybody having failed. A commitment is a promise, and missing it is a failure.

Where an organisation treats predictions as commitments, estimators learn to give commitments instead: numbers padded to be safe, which are then treated as predictions and planned against, which is the worst combination available.

The one estimate that is usually right

Whether something is possible at all. Practitioners are reasonably good at recognising that a thing cannot be done as described, and reasonably bad at saying how long the version that can be done will take.

Which suggests using the judgement where it is reliable: ask whether the shape is right and what the risky part is, and treat the duration as a separate question answered from records rather than from the same conversation.

What to record if you record nothing else

The date a piece of work was expected to be finished, and the date it was. Two columns, kept for a year.

Everything else in this entry follows from having them, and no published figure substitutes for them, because the sources of delay in an organisation are specific to that organisation and mostly have nothing to do with programming.

What a range does and does not fix

Giving a range rather than a number is better and is routinely defeated. The recipient takes the lower end, or averages, or asks which one you really mean.

A range survives only where the recipient understands what the ends mean and plans against the upper one, which is a property of the organisation rather than of the estimate.

What we cannot verify

The work on prediction generally is published and well known. Software-specific accuracy studies vary enormously and rest on datasets whose collection methods differ; the largest of them is discussed elsewhere in this part and is contested. No figure for typical overrun is reproduced here because none of the available ones would transfer.