Budget 2 - Pure Magic 

(Please read the entry “Caveat lector” first)

The standard deviation of a sum of independent, normally distributed random variables is the square root of the sum of the random variables´ variances:

To quote Robbie Williams: “Thoughts running through my head” while reading this will likely be a selection of (a) This guy must be out of his mind (b) Last time I came across this stuff was 20 years ago and I did barely understand it then, let alone today (c) Why, on earth, does anyone want to know (d)… or a sack of rice drops somewhere in China (e) All of the above.

I sympathize with you. It is quite striking how little direct use can be made of much of the material they bother you with at school. Yet every-once-in-a-while you hit a counterexample as if to prove the general validity of the above rule. Sometimes it even turns out to be a shine of light when dealing with messy, real life problems.

Here is one. Let 's suppose the Head of IT is summoned to the Executive Board to present his cost estimate of a large-scale IT project that appears to be unavoidable.

Head of IT: “So, to summarize, after consulting with my team and our external advisors and after going through all the details myself, I estimate the total costs of the envisaged project to be roughly €180 million.”

CEO: “Hmmh. What do you mean by ‘roughly’?”

Head of IT: “Well, any estimate is obviously that, an estimate. There is a significant degree of uncertainty around the number and the actual value will most likely be off by quite some amount.”

CEO: “So, what then is the range for the project cost which you can give us?”

Head of IT: “Well, hard to say. But as we do not have detailed all requirements and due to the fact that the project takes several years to complete, I would assume this to be my estimate plus or minus 30%”.

CEO frowns and throws a menacing glance at the CIO.

CIO: “Entirely plausible. This is about as accurate as you can be at this stage.”

CEO looking back at the Head of IT: “In other words, the range is between, what, €126 million and €234 million?”

Head of IT: “Like I said, estimate +/- 30%”. Obviously, we´ll get a better read once we start with the project”.

As a consequence, the Executive Board, with very little domain expertise itself, and in the dire position of having to face the Supervisory Board soon, is left with an enormous cost range to defend, even though the +/- 30% value by itself seems intuitively okay, given the nature of the uncertainties involved.

I leave it to your imagination what the typical conclusion will be for the meeting of the Supervisory Board. But let ´s go back to the awkward formula at the beginning of this entry. Let us assume that the Head of IT´s +/- 30% range estimate is in fact the best one can do to be 95% confident that the actual value will turn out to be inside that range.

Now suppose that we could divide the project into 9 different subprojects. Each one is planned and analyzed independently. As it turns out (by coincidence), all subprojects have the same expected costs, i.e. €20 million and the uncertainty around each estimate again is +/- 30%, i.e. the range estimate per subproject is between €14m and €26m.

The estimate for the total costs of the project as a whole is simply the sum of the estimates of the subprojects: 9 * €20m = €180 million. Exactly the same number we got from the first overall estimation exercise.

Now, what about the range figure for the sum of the subproject estimates? This is the very moment where a blip of statistical magic happens. If we assume that the subproject estimates are truly independent of each other and unbiased we may apply the formula at the beginning of this blog entry and the comparable range figure (95% confidence level) for the sum of all subprojects is +/- €18 million. Et voila!!

To put this in perspective: The original estimate for the project was 

€180m +/-30% (i.e. +/- €54m) yielding a ('hopeless') range of €126-234m. 

The “sum-of-subproject-estimate”, by contrast, has the same expected cost value of 

€180m but only +/- €18m yielding a range of €162-198m 

and still coming down with the same 95% confidence level! I guess it is fair to assume that the type of discussion at the Supervisory Board meeting will be very different under these circumstances.

The reason for this quite astonishing result is the fact that you have independent estimates of the subprojects and therefore, there is no covariance between those independent results. Now, are the subprojects truly “independent, normally distributed random variables”? Well, most certainly not. At least not in the ideal sense. However, they do not have to be in order to enable a dramatic (non-linear) reduction of the cost range estimate. All you need to do when having to calculate a cost range estimate for a large-scale IT project is to divvy the overall program up into as many more or less self-contained chunks as possible, have those chunks be independently estimated, and then add them all up applying the formula above. Even if some level of covariance between the individual estimates cannot be excluded, the 95% confidence range for the overall program estimate will shrink substantially.

The fun truly starts once you get to apply this in reverse. Let ´s say the Head of IT, when questioned, claims: "But we did all that! We have defined and estimated subprojects! But the result for the overall program still is +/-30%.", looking around the table for approval. If you then do the math (well, arithmetic) in our little example the implied estimate range for one of the subprojects turns out to be €2m-€38m. Hmmh! Unless there is a rather peculiar reason for this, chances are you may have an IT problem to solve, but the more urgent one seems to be HR.

So, once you have broken down your large-scale IT project and analyzed it as described above, you should feel confident about your figures, enjoy even a short-lived moment of respect from your superiors, but please, by all means, do not forget to include a contingency budget in your overall estimate! I am certain, advice on how to do this properly is not far away ;-)

(all rights reserved)