AuthorsD. Pradhan, S. Wang, T. Yue, S. Ali and M. Liaaen
TitleAutomated Test Case Implantation to Test Untested Configurations: A Cost-Effective Search-Based Approach
AfilliationSoftware Engineering
Project(s)The Certus Centre (SFI), Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines 
StatusPublished
Publication TypeTechnical reports
Year of Publication2018
PublisherSimula Research Laboratory
KeywordsGenetic Algorithm, Multi-objective optimization, Search, test case implantation
Abstract

Modern large-scale software systems are highly configurable, and thus require a large number of test cases to be implemented and revised for testing a variety of system configurations. This makes testing highly configurable systems very expensive and time-consuming. Driven by our industrial collaboration with a video conferencing company, we aim to automatically analyze and implant existing test cases (i.e., an original test suite) to test the untested configurations. We propose a search-based test case implantation approach (named as SBI) consisting of two key components: 1) Test case analyzer that statically analyzes each test case in the original test suite to obtain the program dependence graph for test case statements and 2) Test case implanter that uses multi-objective search to select suitable test cases for implantation using three operators, i.e., selection, crossover, and mutation (at the test suite level) and implants the selected test cases using a mutation operator at the test case level including three operations (i.e., addition, modification, and deletion). We empirically evaluated SBI with an industrial case study and an open source case study by comparing the implanted test suites produced by SBI with the original test suite using evaluation metrics such as statement coverage (SC), branch coverage (BC), mutation score (MS). Results show that for both the case studies, the implanted test suites performed significantly better than the original test suites with on average 21.9% higher coverage of configuration variable values. For the open source case study, SBI managed to improve SC, BC, and MS with 4.8%, 7.5%, and 2.6%, respectively. SBI can be applied to automatically implant an existing test suite with the aim of testing untested configurations and thus achieving higher configuration coverage. 

Notes

This research was supported by the Research Council of Norway (RCN) funded Certus SFI. Shuai Wang is also supported by RFF Hovedstaden funded MBE-CR project and COST Action CA15140 (ImAppNIO). Tao Yue and Shaukat Ali are supported by RCN funded Zen-Configurator project, RCN funded MBTCPS project, and COST Action IC1404. 

URLhttps://www.simula.no/sites/default/files/publications/files/automated_test_case_implantation_to_test_untested_configurations-_a_cost-effective_search-based_approach_tr.pdf
Citation Key26017