An Innovative Approach to Quantifying Uncertainty in Early Lifecycle Cost Estimation by Jim McCurley & Bob Stoddard
Extensive cost overruns are common and often associated with optimistic expectations about achievable program scope and technology that can be delivered on schedule and within budget. The problem is exacerbated by the fact that the estimates at the beginning of a program incorporate a great deal of uncertainty about large-scale, unprecedented systems that take years to develop and deploy. Needed capabilities and yet-to-be-developed technical solutions are not yet well understood. Early estimates, by necessity, rely heavily on expert judgment. Many assumptions about the desired end product are made in calculating the estimates. The estimation process does not capture information about program change factors that can dramatically influence cost over the lifecycle of program research, development, production, deployment and sustainment.
In this presentation we describe a new, integrative approach we call QUELCE (Quantifying Uncertainty in Early Lifecycle Cost Estimation). QUELCE synthesizes scenario building, Bayesian Belief Network (BBN) modeling and Monte Carlo simulation into an estimation method that quantifies uncertainties, allows subjective inputs, visually depicts influential relationships among change drivers and outputs, and assists with the explicit description and documentation underlying an estimate. We use scenario analysis and design structure matrix (DSM) techniques to limit the combinatorial effects of multiple interacting program change drivers to make modeling and analysis more tractable. Representing scenarios as BBNs enables sensitivity analysis, exploration of alternatives, and quantification of uncertainty. The BBNs and Monte Carlo simulation are then used to predict variability of what become the inputs to existing, commercially available methods and tools in use by cost estimators. As a result, interim and final cost estimates are embedded within clearly defined confidence intervals.
QUELCE aims to provide credible and accurate program cost estimates within clearly defined, statistically valid confidence intervals. By making visible the potential changes that may occur during program execution, our approach also supports quick revision of program estimates to mitigate risk and respond more quickly to the changes that often arise over a program’s lifecycle. The same flexibility enables early consideration of the likely impact of different possible future scenarios on the estimates. Intuitive visual representations of the data explicitly model influential relationships and interdependencies among the drivers on which the ultimate estimates depend. Assumptions and constraints underlying the estimates that may not otherwise have been considered are well documented, which contributes to better management of cost, schedule, and adjustments to program scope as more is learned and conditions change. Documenting the basis of an estimate also facilitates updating the estimate during program execution and helps others make informed judgments about estimation accuracy.