AuthorsH. C. Benestad, E. Arisholm and D. I. K. Sjøberg
EditorsH. E., M. B.E., R. K. and F. L.S.
TitleHow to Recruit Professionals As Subjects in Software Engineering Experiments
Afilliation, Software Engineering
Publication TypeProceedings, refereed
Year of Publication2005
Conference NameIRIS (Information Systems Research in Scandinavia), August 6-9, Kristiansand Norway
Date PublishedAugust
PublisherDepartment of Information Systems, Agder University College

Controlled experiments are the classical scientific method for identifying cause-effect relationships, and are complementary to case studies and surveys as a means to empirically evaluate information systems development methods and practices. Most controlled experiments that evaluate software development methods and practices use students as subjects. Using students as subjects is convenient. However, a common criticism of controlled experiments in which students are used as subjects is that it is difficult to generalize the results to industrial settings. Consequently, Simula Research Laboratory has included professionals as subjects in software engineering experiments. At present, more than 750 professional software developers from 46 software development organizations have participated in our experiments. From this experience we have identified three important principles for research groups that want to include professional software developers as subjects in controlled experiments. First, practical constraints must be considered when defining the target population of software developers. Second, the participating organizations must be offered flexibility and value using a planned communication strategy, in order to ensure adequately sized representative samples of organizations and individuals. Third, to ensure long-term, relationships with the organizations, high professional and ethical standards must be employed.

Citation KeyBenestad.2005.1