|Authors||D. Pradhan, S. Wang, T. Yue, S. Ali and M. Liaaen|
|Title||Automated Test Case Implantation to Test Untested Configurations: A Cost-Effective Search-Based Approach|
|Project(s)||The Certus Centre (SFI), Zen-Configurator: Interactive and Optimal Configuration of Cyber Physical System Product Lines|
|Publication Type||Technical reports|
|Year of Publication||2018|
|Publisher||Simula Research Laboratory|
|Keywords||Genetic Algorithm, Multi-objective optimization, Search, test case implantation|
Modern large-scale software systems are highly configurable, and thus require a large number of test cases to be implemented and revised for testing a variety of system configurations. This makes testing highly configurable systems very expensive and time-consuming. Driven by our industrial collaboration with a video conferencing company, we aim to automatically analyze and implant existing test cases (i.e., an original test suite) to test the untested configurations. We propose a search-based test case implantation approach (named as SBI) consisting of two key components: 1) Test case analyzer that statically analyzes each test case in the original test suite to obtain the program dependence graph for test case statements and 2) Test case implanter that uses multi-objective search to select suitable test cases for implantation using three operators, i.e., selection, crossover, and mutation (at the test suite level) and implants the selected test cases using a mutation operator at the test case level including three operations (i.e., addition, modification, and deletion). We empirically evaluated SBI with an industrial case study and an open source case study by comparing the implanted test suites produced by SBI with the original test suite using evaluation metrics such as statement coverage (SC), branch coverage (BC), mutation score (MS). Results show that for both the case studies, the implanted test suites performed significantly better than the original test suites with on average 21.9% higher coverage of configuration variable values. For the open source case study, SBI managed to improve SC, BC, and MS with 4.8%, 7.5%, and 2.6%, respectively. SBI can be applied to automatically implant an existing test suite with the aim of testing untested configurations and thus achieving higher configuration coverage.
This research was supported by the Research Council of Norway (RCN) funded Certus SFI. Shuai Wang is also supported by RFF Hovedstaden funded MBE-CR project and COST Action CA15140 (ImAppNIO). Tao Yue and Shaukat Ali are supported by RCN funded Zen-Configurator project, RCN funded MBTCPS project, and COST Action IC1404.