AuthorsM. Jørgensen, K. Teigen and K. J. Moløkken-Østvold
TitleBetter Sure Than Safe? Overconfidence in Judgment Based Software Development Effort Prediction Intervals
AfilliationSoftware Engineering, Software Engineering
Publication TypeJournal Article
Year of Publication2004
JournalJournal of Systems and Software

The uncertainty of a software development effort estimate can be indicated through a prediction interval, i.e., the estimated minimum and maximum effort corresponding to a specific confidence level. For example, a project manager may be “90% confident” or believe that is it “very likely” that the effort required to complete a project will be between 8,000 and 12,000 work-hours. This paper describes results from four studies (Studies A-D) on human judgement (expert) based prediction intervals of software development effort. Study A examines the accuracy of the prediction intervals in real software projects. The results suggest that the prediction intervals were generally much too narrow to reflect the chosen level of confidence, i.e., that there was a strong over-confidence. Studies B, C and D try to understand the reasons for the observed over-confidence. Study B examines the possibility that the over-confidence is related to type of experience or estimation process. Study C examines the possibility that the concept of confidence level is difficult to interpret for software estimators. Finally, Study D examines the possibility that there are unfortunate feedback mechanisms that reward over-confidence.

Citation KeySE.4.Joergensen.2004.a