AuthorsE. Rogstad and L. Briand
TitleCost-effective Strategies for the Regression Testing of Database Applications: Case study and Lessons Learned
AfilliationSoftware Engineering, Software Engineering
StatusPublished
Publication TypeJournal Article
Year of Publication2016
JournalJournal of Systems and Software
Volume113
Pagination 257–274
PublisherSimula Research Laboratory
KeywordsClassification tree modeling, Combinatorial testing, Database applications, Operational profile testing, Regression testing, Test data generation
Abstract

Testing and, more specifically, the regression testing of database applications is highly challenging and costly. One can rely on production data or generate synthetic data, for example based on combinatorial techniques or operational profiles. Both approaches have drawbacks and advantages. Automating testing with production data is impractical and combinatorial test suites might not be representative of system operations.

In this paper, based on a large scale case study in a representative development environment, we explore the cost and effectiveness of various approaches and their combination for the regression testing of database applications, based on production data and synthetic data generated through classification tree models of the input domain.

The results confirm that combinatorial test suite specifications bear little relation to test suite specifications derived from the system operational profile. Nevertheless, combinatorial testing strategies are effective, both in terms of the number of regression faults discovered but also, more surprisingly, in terms of the importance of these faults. However, our study also shows that relying solely on synthesized test data derived from test models could lead to important faults slipping to production. Thus, we recommend that testing on production data and combinatorial testing be combined to achieve optimal results. 

Citation Key24413