AuthorsM. Jørgensen and S. Grimstad
TitleThe Impact of Irrelevant and Misleading Information on Software Development Effort Estimates: a Randomized Controlled Field Experiment
AfilliationSoftware Engineering, Software Engineering
StatusPublished
Publication TypeJournal Article
Year of Publication2011
JournalIEEE Transactions on Software Engineering
Volume37
Number5
Pagination695-707
Abstract

Several studies have reported that software development effort estimates can be strongly affected by effort-irrelevant and misleading information without the estimators being aware of this effect. These studies were conducted in laboratory (artificial) estimation contexts. To increase our knowledge about the importance of these effects in field settings, we paid 46 outsourcing companies from Eastern European and East Asian countries to estimate the required effort of the same five software development projects. The companies were allocated randomly to either the original requirement specification or a manipulated version of the original requirement specification. The manipulations were as follows: i) reduced length of requirement specification with no change of content, ii) information about the low effort spent on the development of the old system to be replaced, iii) information about the client's unrealistic expectations about low cost, and iv) a restriction of a short development period with start up a few months ahead (which should, rationally speaking, lead to an increase in effort). All manipulations led to decreased median effort estimates, but only manipulation iv) led to a large, statistically significant decrease. A comparison of the effects of similar types of irrelevant and misleading information in laboratory and field settings suggests that the effect of manipulations i), ii) and iii) where much lower in field settings than in laboratory settings, while the effect of manipulation iv) was almost at the same level. We conclude that the tendency towards a smaller effect in field settings means that laboratory studies are frequently only useful for demonstrating the existence of a software engineering phenomenon, or for understanding it better, and that we need field studies to analyze its importance.