IMPROVING THE PROCESS OF COST ESTIMATION Lbj
where). Thus, if the MAD is a sizable fraction of the variable being estimated, the average error is large and the forecast or estimate is not very accurate.
Now, consider Column D. The sum of the entries in this column for any number of periods is the sum of the forecast errors, often referred to as the "running sum of the forecast errors" (RSFE). If the estimator's errors are truly random, their sum should approach zero; that is, the RSFE should be a small number because positive errors should be offset by negative errors. If either positive or negative errors are more numerous or consistently larger than the other, the estimation process is said to be biased and the errors are not random. In Figure 7-5, RSFE = 133, so .the forecast is quite positively biased.
The tracking signal measures the estimator's bias. It is easily found:
Note that it calculates the number of MADs in the RSFE (see column G in Figure 7-5, and recall the similarity between MAD and standard deviation). If the RSFE is small, approaching zero, the TS will also approach zero. As the RSFE grows, the TS will grow, indicating bias. Division of the RSFE by the MAD creates a sort of "index number," the TS, that is independent of the size of the variables being considered. We cannot say just how much bias is acceptable in an estimator/forecaster. We feel that a TS s 3 is too high unless the estimator is a rank beginner. Certainly, an experienced estimator should have a much lower TS. (It should be obvious that the TS may be either negative or positive. Our comment actually refers to the absolute value of theTS.)
Perhaps more important than worrying about an acceptable limit on the size of the tracking signal is the practice of keeping track of it and analyzing why the estimator's bias, if there is one, exists. Similarly, the estimator should consider how to reduce the MAD, the average estimation error. Such analysis is the embodiment of "learning by experience." The Lotus 1-2-3® template makes the analysis simple to conduct, and should result in descreasing the size of both the MAD and the TS. (For those familiar with Lotus 1-2-3®, the formulas used for Figure 7-5 are shown in Figure 7-6.)
Some estimators would like to speed up the process of improving their estimation skills by grouping forecasts of different resources to generate more data points when calculating their MADs and TSs. Use of the tracking signal requires that the input data, estimates (forecasts) and actuals, be collected and processed separately for each variable being estimated. Cost estimates and actuals for different resources, for instance, would be used to find the MAD and TS for each individual resource. The reason for this inconvenience is that resources come in different units and the traditional caution about adding apples and oranges applies. (Even if all resources are measured in dollars, we still have scale problems when we mix resource costs of very different sizes.) Fortunately, there is a way around the problem.
Instead of defining the estimation error as the difference between actual and forecast, we can define it as the ratio of actual to forecast. Therefore, the new error for the first forecast (Period 1) in Figure 7-5 is not 8 units, but rather is
Post a comment