AuthorsG. Fraser and A. Arcuri
TitleEfficient Mutation Testing Using Whole Test Suite Generation
AfilliationSoftware Engineering, Software Engineering, Software Engineering
Project(s)The Certus Centre (SFI)
Publication TypeProceedings, refereed
Year of Publication2013
Conference Nameconference
Place Published.

By seeding artificial faults (mutants), mutation testing can tell us how good existing tests are, and it can help to direct test generation efforts. However, mutation based test generation is hampered because there usually are simply too many mutants, and too many of these mutants are either trivially killed or equivalent. Any effort spent on test generation for equivalent mutants is per definition wasted, and misdirects resources from where they could be put to better use - for example to produce tests revealing more non-equivalent mutants. To overcome this problem, our search-based EVOSUITE test generation tool integrates two optimizations: First, we avoid redundant test executions on mutants by monitoring state infection conditions, and second we use whole test suite generation to optimize test suites towards killing the highest number of mutants, rather than selecting individual mutants. These optimizations allowed us to perform one of the largest empirical studies on mutation testing to date, where we applied EVOSUITE to a random sample of 100 open source projects consisting of a total of 8,963 classes, leading to a total of 1,380,302 mutants. The experiment not only demonstrates that our approach scales well, but it also quantifies the relations between weak and strong mutation testing as well as branch coverage, and points out current limitations.

Citation KeySimula.simula.1550