AuthorsE. Arisholm, S. A. Ali and S. E. Hove
TitleAn Initial Controlled Experiment to Evaluate the Effect of UML Design Documentation on the Maintainability of Object-Oriented Software in a Realistic Programming Environment
Publication TypeTechnical reports
Year of Publication2003
PublisherSimula Research Laboratory

The Unified Modeling Language (UML) is becoming the standard notation for expressing object-oriented analysis and design models. However, to date, there are only very few empirical studies that have attempted to evaluate UML, that is, to evaluate the potential costs, benefits and appropriate use of UML. For example, a common claim is that the UML documentation will improve the ease of understanding how to perform changes to the software it represents. However, this claim has not been evaluated empirically. Furthermore, using UML may incur additional costs, because the UML documents need to be developed and maintained. Thus, the tradeoffs between the costs and benefits of using UML need to be investigated. This paper describes a controlled experiment to evaluate the effect of UML documentation (versus no UML documentation) on the maintainability of object-oriented software. In this experiment, 20 students designed and coded several Java change (maintenance) tasks. About half of them received UML documentation, whereas the other half did not. The subjects who received UML documentation also had to update the documentation using a tool, Tau UML. The primary goal of the experiment was to evaluate to which extent the access to UML class and sequence diagrams improves the ease of understanding and changing object-oriented software. To assess the costs and benefits of using UML in object-oriented development projects, the experiment also attempted to assess the additional costs associated with updating the UML documentation. The dependent variables of the study were effort (in minutes spent to solve the tasks) and correctness. The effort data was reported by each subject using a task questionnaire. The correctness of the task solutions was assessed by the researchers. The subjects spent between five and eight hours solving tasks. Five observers were present during the experiment. A secondary goal of the experiment was to evaluate how a so-called “think-aloud screen” affected the performance of subjects in controlled software engineering experiments, that is, whether the think-aloud screen decreases performance of the subjects (task solving time and correctness of the task solutions) compared with the silent condition, and whether the think-aloud screen provides valuable information that cannot be collected in the silent condition. This paper focuses on the evaluation of the effect of using UML to understand and change object-oriented software. Further discussion of the effects of using the think-aloud screen is discussed in [5].

Citation KeySE.7.Arisholm.2003.c