AuthorsA. Karahasanovic, A. Fjuk, D. I. K. Sjøberg and R. Thomas
TitleA Controlled Experiment to Evaluate the Reactivity and Usefulness of the Think-Aloud Tool
AfilliationSoftware Engineering, Software Engineering
Publication TypeProceedings, refereed
Year of Publication2004
Conference NameInformation Resources Management Association International Conference (IRMA'04), Human Computer Interaction Track
Date PublishedMay 23-26
PublisherIdea Group Publishing
Place PublishedNew Orleans, Louisiana, USA

This paper presents ongoing work on the evaluation of a tool for collecting feedback in large-scale software engineering and program comprehension experiments. Two most important questions one can ask about a new verbal procedure are whether it is useful and whether it is reactive. We have developed a tool, called the think-aloud tool, to support the think-aloud method in a variety of experiments. This tool instructs the subjects at regular intervals to write down their thoughts on a web based screen, as opposed to the oral expression of thoughts in conventional think-aloud method. The tool collects the subjects' feedback together with timestamps. This paper describes an experiment designed to investigate whether the think-aloud tool affects performance of the subjects and whether it collects information beyond that collected in traditional think-aloud and retrospective reports. Furthermore, we will conduct a qualitative comparison of mouse and keying rates with think-aloud and retrospective reports. In addition, the experiment aims to check for any effect from using the think-aloud tool on the subject's task-related understanding and problem solving.

Citation KeySE.5.Karahasanovic.2004