AuthorsD. Pradhan, S. Wang, S. Ali, T. Yue and M. Liaaen
TitleSTIPI: Using Search to Prioritize Test Cases based on Multi-Objectives Derived from Industrial Practice
AfilliationSoftware Engineering, The Certus Centre (SFI), Software Engineering
Publication TypeProceedings, refereed
Year of Publication2016
Conference NameThe 28th International Conference on Testing Software and Systems (ICTSS)
Date Published10/2016
PublisherLecture Notes in Computer Science, Springer Verlag
Place PublishedGraz, Austria

The importance of cost-effective test case prioritization is undeniable in automated testing practice in industry. Such prioritization typically relies on various cost and effective objectives. This paper focuses on prioritizing test cases developed to test product lines of Video Conferencing Systems (VCSs) at Cisco Systems, Norway. Each test case requires setting up configurations of a set of VCSs, invoking a set of test APIs with specific inputs, and checking the status of the VCSs. Based on these characteristics and information available about the execution of test cases (e.g., number of faults detected), we identified that the test case prioritization problem in our particular context should focus on achieving high coverage of configurations, test APIs, statuses, and high fault detection capability as fast as possible. We propose a search-based test case prioritization approach (named STIPI) to solve this problem by defining a fitness function with four objectives and integrating it with the widely applied multi-objective Non-dominated Sorting Genetic Algorithm II. We compared STIPI with random search (RS), Greedy algorithm, and three approaches adapted from literature, using three real sets of test cases from Cisco with four time budgets (25%, 50%, 75% and 100%). Results show that STIPI significantly outperformed the selected approaches and managed to achieve better performance than RS for on average 39.9%, 18.6%, 32.7% and 43.9% for the coverage of configurations, test APIs, statuses and fault detection capability, respectively.

Citation Key24632