基于混和效应移除的测试集有效性预测模型及其应用研究

This project aims to develop a confounding-effect-removal based model for predicting the effectiveness of test suites in detecting faults and to explore its applications in regression testing. Currently, code coverage ratio and mutation score are the most commonly used methods to evaluate test suite effectiveness. However, they do not take into account the influence of many confounding factors such as test suite size on test suite effectiveness. Consequently, test suite effectiveness might be overestimated. To attack this problem, this project will first identify the possible confounders and then build statistical confounding effect models to quantify their influence on the relationships between code coverage ratio/mutation score and test suite effectiveness. After that, this project will propose a method to remove the confounding effect of these confounders and use the cleaned coverage ratio/mutation score to predict test suite effectiveness. The research contents of this project are listed as follows: (1) development of the prediction models for test suite effectiveness, including identifying confounders, modeling confounding effect, removing confounding effect, and building the prediction model; (2) application of the proposed models to regression testing, including test case prioritization, test suite reduction, test case selection, and defect prediction; and (3) empirical validation on real-world software. The outputs of the project will promote the theoretic development of software testing techniques as well as their applications in practice.

本项目旨在移除混和效应的基础上建立测试集有效性预测模型，以客观评价测试集的缺陷检测能力，并探索它在回归测试中的应用。代码覆盖率和变异得分是当前最常用的测试集有效性评价指标，但它们没有考虑测试集规模等混和因素的影响，有可能高估测试集的测试有效性。为此，本项目首先系统地识别影响测试集测试有效性的混和因素，然后利用统计方法建立混和效应模型刻画混和因素对代码覆盖率/变异得分与测试有效性之间关系的影响，之后提出统计方法移除混和效应，最后据此建立测试集有效性预测模型并在回归测试中进行应用。主要研究内容为：（1）测试有效性预测模型的理论研究，包括混和因素识别、混和效应建模、混和效应移除以及测试有效性评价模型构建；（2）测试有效性预测模型在回归测试中的应用研究，包括测试用例优先级、测试集约简、测试用例选择和缺陷模块预测；（3）结合实际应用展开实验验证。本项目能促进软件测试技术的理论研究和实际应用。

本项目研究测试集有效应评估技术及其应用，目标是提出能客观评估测试集有效性的技术并探索其应用。在项目执行过程中，项目组成员深入学习了变异测试、测试集有效性度量、统计分析和机器学习技术等内容，在此基础上深入研究了测试集有效性预测模型，主要包括混和效应的建模与分析、变异得分预测、变异约简评价、测试集有效性度量准确性评价等关键内容，提出了相应的解决方案并对其有效性进行了实验评估，完成了项目计划的研究要点。项目的研究成果有助于准确刻画测试集的有效性，从而推动软件测试技术的发展。