概化理论多侧面设计缺失数据方差分量及其变异量估计

Generalizability theory is widely applied in psychological measurement and evaluation. The variability of estimated variance component, which is constrained by sampling, is the "Achilles heel" of generalizability theory. Therefore, estimating the variability of estimated variance components needs to be further explored. Most researchers only focused on full data and often ignored missing data. But missing observation are common in psychological surveys and mental experiments. Because these assessments are time-consuming to administer and score, examinees seldom respond to all test items and raters seldom evaluate all examinee responses. As a result, a common problem encountered by those using generalizability theory with large-scale performance assessments is working with missing data. The data from such examinations compose a missing data matrix. Researchers have to be usually concerned about how to make good use of limited missing data. As for these missing data, researchers usually delete them or make an interpolation for missing records, but some problems may be caused in following aspects. First, it is not very effective if doing it and can not carry out some statistical analysis. Second, there are many different rules of interpolation, but people do not know which rules are unbiased. Because of missing data, a series of problems may be caused when estimating variance component of unbalanced data has to be done for generalizability theory. To solve these above problems about estimating variance components and their variability of missing data of multifaceted designs for generalizability theory, the study will make an innovation from three aspects as follows. First, we will develop multifaceted designs from single faceted p×i design to p×i×r design and p×(i:r) design that are two faceted designs and are more complicated. Based on generalizability theory, it is a key issue how to effectively use the existing missing data to do more statistical analysis for multifaceted designs. Second, we will investigate some methods which do not need to adopt deleting the records or adopt the method of interpolation for missing values. As for these developed methods, we also will compare them in order to find which of four ways can estimate the variance component and their variability of missing data rapidly and effectively. As for estimating variance components, there are four methods such as formulas method, restricted maximum likelihood estimation method, subdividing method and Markov Chain Monte Carlo (MCMC) method. As for estimating the variability of variance components, there are four methods such as traditional method, jackknife method, bootstrap method and MCMC method. Finally, based on these four methods, the study will adopt Monte Carlo data simulation technique to compare estimated variance components and their variability. At the same time, the study will combining the data of simulation and real data to further validate our conclusion.

概化理论广泛应用于心理测评实践中。方差分量估计是进行概化理论分析的关键。方差分量估计受限于抽样，需要对其变异量进行探讨。多数学者仅关注概化理论完备数据的方差分量及其变异量估计，却对缺失数据视而不见。但各种心理调查与心理实验中，数据的缺失随处可见。人们通常采用删除缺失记录或对缺失值进行插补处理，但弊端是不能充分利用现有数据进行方差分量及其变异量估计，可供分析的数据大量减少。为了解决概化理论缺失数据方差分量及其变异量估计的问题，本研究拟从以下三个方面进行创新：第一，将单侧面p×i设计发展到双侧面p×i×r和p×(i:r)设计。第二，探讨不需要删除记录或对缺失值进行插补的方法，并比较这些方法的性能优劣。对于方差分量估计，包括公式法、拆分法、REML法和MCMC法，对于方差分量变异量估计，包括传统法、Jackknife法、Bootstrap法和MCMC法。第三，将数据模拟技术与实际数据验证相结合。

在各种心理调查、心理实验中，数据缺失随处可见。例如，在考试测验中，出于时间、人力、物力考虑，无法做到所有评分者对所有试卷进行评分。概化理论广泛应用于心理测评实践中。多数学者仅关注概化理论完备数据的方差分量及其变异量估计，却对缺失数据视而不见。.本项目旨在研究概化理论多侧面设计缺失数据方差分量及其变异量估计，主要内容包括：第一，自行推导含缺失数据方差分量及其变异量公式，将单侧面p×i设计发展到双侧面p×i×r和p×(i:r)设计。第二，探讨不需要删除记录或对缺失值进行插补的方法，并比较这些方法的性能优劣。对于方差分量估计，包括公式法、拆分法、REML法和MCMC法，对于方差分量变异量估计，包括传统法、Jackknife法、Bootstrap法和MCMC法。第三，将数据模拟技术与实际数据验证相结合。.本项目重要结果如下：第一，MCMC方法估计多侧面设计缺失数据方差分量，较其它三种方法表现出更强的优势。MCMC方法不存在传统法估计某些方差分量偏差较大的情形，也不存在REML法迭代不收敛的情况，也无需类似于拆分法将方差分量合并。第二，Bootstrap方法估计方差分量变异量较传统法、Jackknife法和MCMC方法好。.本项目关键数据如下：第一，没有任何一种方法同时估计方差分量和方差分量变异量都好，为了准确估计方差分量及其变异量，必须同时使用两种以上方法。第二，题目和评分者是缺失数据方差分量及其变异量估计重要的两种影响因素，在人力物力有限的情况下，可优先考虑增加题目数量。.本项目科学意义如下：第一，引导人们注重考查缺失数据，这有利于克服大多数研究者仅关注概化理论完备数据的局限；第二，厘清缺失数据方差分量点估计与变异量估计存在的不同，MCMC方法对前者估计有效，而Bootstrap方法却对后者估计有效，促使人们反思概化理论缺失数据估计方法与方差分量及其变异量估计之间存在“交互效应”。