Courses-summer-schools
Empirical Research Methods in Software Engineering and Informatics

Empirical Research Methods in Software Engineering and Informatics

This intensive course was held in September 2022.

Key Details:

Dates: 26 to 30 September 2022

Target group: The course is intended for people that have little-to-no background in empirical research methods, in particular people with a technical or formal methods background. The ideal participant has recently started their PhD in Computer Science / Software Engineering, though more junior and senior researchers are welcome.

Location: Simula Research Laboratory, Oslo, Norway.

Motivation and Description:

Traditionally, Software Engineering and Informatics have been strongly influenced by Mathematics. Empirical Research has always been around, too, but has gained traction only in the past 10-15 years. Today, empirical research represents a major approach that is highly visible in the most important conferences and journals in the field. Many reviewers will demand empirical evidence when reviewing papers making statements about the reality world that could be tested empirically. Yet, in education empirical research methods still are underrepresented.

This course aims to provide students with an overview and concrete starting point, possibly removing any inhibitions or hesitation that might be there. We aspire to equip participants with some skills and first practical exercises in conducting methodologically sound research. After attending this course, participants:

  • are capable of choosing an appropriate research paradigm for a given problem;
  • will be aware of the potential and limitations of empirical research methods; and
  • can assess the quality of the empirical research reported in an article, such as for a review.

We will cover controlled experiments, selected qualitative methods, and overall study design as far as the practical exercises are concerned. We explicitly encourage participants to bring along their own empirical research project to work on them in the setting of this course. Time permitting, we will discuss these projects also in plenary sessions.

This course has been running in different shapes and forms for ten years now.

Course content and Program

The course introduces you to empirical research methods in a very practical, hands-on way. There will be introductory lectures on techniques followed by group work on exercises, for instance:

  • analyze the study design in a given article with a view to finding shortcomings and threats to validity and how they impact the claims put forward;
  • design a study yourself for a given case study in a small group, and present the study design in a plenary discussion;
  • develop a research question and discuss alternative approaches to providing evidence for or against it.

The program will cover the following topics during the week:

  • Controlled Experiements, Threats to Validity, Physiological Measurements, Statistical Techniques
  • Qualitative Methods (Interviews, Surveys), Observational Techniques, Grounded Theory, Ethnography
  • Secondary Research, Mapping Studies
  • Structured Literature Reviews, Tertiary Studies
  • Philosophical Perspective, Observations and Measurements, Wrap-Up and Feedback

About the Instructors

Harald Störrle

Harald Störrle received a Dipl.-Inform. and a Dr.rer.nat. from the Universities of Hamburg (1997) and Munich (2000), respectively. From 2001 to 2009 he worked as a software architect and methodology consultant in industry, sidelining as an adjunct lecturer at the University of Munich. Starting 2006, he held lecturer positions at the Universities of Innsbruck and Munich, and as an Associate Professor of Software Engineering at the Technical University of Denmark (DTU) in Lyngby near Copenhagen. Since 2017, he has worked as a principal consultant with QAware in Munich.

In his research activities, Harald applies empirical research methods to problems in modelling, requirements, and software processes. He advocates Evidence Based Software Engineering, favouring methodological openness and diversity.

He is a Senior Member of the ACM, elected member of the ACM Europe Council, former vice chairperson of the German Chapter of the ACM, and was appointed to several ACM boards and committees, including the ACM practitioner board.

Jefferson Seide Molléri

Jefferson Molléri holds a PhD in Software Engineering from the Institute of Technology of Blekinge (2019) - Sweden. Currently, he is a Postdoctoral Researcher in Information Technology Management at Simula Metropolitan. He acted as a university lecturer in both first- and second-cycle courses in software engineering and computer science programs. He has over ten years of industrial experience in companies developing web-based and client-server applications.

As an academic researcher, Jefferson focuses on evidence-based software engineering and empirical research methods. He stands up for methodological quality, ethical and contributory research. Among his contributions to the field, he has published a report on empirical standards, a catalogue of empirical research guidelines, and checklists for assessing survey research.