多级评分认知诊断的测验设计、模型开发及施测方法研究

Cognitive diagnostic assessment (CDA) can provides potentially useful diagnostic information for educators, parents, and students. We believe that such feedback will help to improve the effect of teaching and learning. CDA has become an exciting new field of psychometric research. To make inferences about students' cognitive skills, the vast majority of cognitive diagnostic models (CDMs) must utilize the Q matrix which is an incidence matrix of the items and skills. Tatsuoka (1995,2009) first introduced the Q-matrix theory. Ding and his colleagues (2011,2012) then improved it by giving a sufficient and necessary condition that guarantees to built a bijection between knowledge states and ideal response patterns for the non-compensative and conjunctive model using dichotomous scoring. And they augment contents of the Q matrix theory by exploring the way to build a GOOD test blueprint for cognitive diagnosis and employing the results to some applications, such as building an item pool, designing a test, constructing a cognitive diagnostic model, designing a strategy of selection item for computerized adaptive testing with cognitive diagnosis (CD-CAT), identifying the attributes being adhesive to a raw item, calibrating unknown parameters of raw item, and so on. Accurate cognitive diagnostic requires a rich description of students' test performance, which makes polytomously-scored test items be of potentially greater value than dichotomously scored items for classifying students according to the probability of attribute mastery. But due to its complexity. little is known about the Q matrix theory for polytomous cognitive diagnosis. Some examples show that there exist some fundamental differences between polytomously and dichotomously scored items when constructing the GOOD test blueprint. So a refined criteria of judging what is a good test blueprint seems necessary for the polytomously scored data. This project seeks to accomplish the following purposes: (1) Constructing the Q matrix theory on polytomous data . (2) Finding the way to explicitly incorporate the Q matrix theory into development of classification and strategies diagnostic methods. (3) Studying and comparing the performance of cognitive diagnostic computerized adaptive testing(CD-CAT), multistage CD-CAT, and classroom-based testing. For example, with regard to classroom assessment, usually there is no item pool so CD-CAT is not applicable, how to give the diagnostic information to students in time is an issue worth discussing. (4) Focusing on how to evaluate the quality of test and how to modify Q matrix after collecting test data in polytomous cognitive diagnosis models. This proposal is the extension of three researches supported by the National Natural Science Foundation under Grant No. 60263005, 30860084,31160203.

认知诊断（CD）能有效促进教学、帮助学习困难者,国内外认知心理学和教育与心理测量工作者对其额外关注。0-1评分（DS）CD的理论和应用研究成果颇丰。多级评分（PS）数据比DS数据提供更多的信息，故可望利用PS数据导出被试知识状态及加工策略的更准确估计。然而或因习惯而不前，或因困难而却步，致使PS-CD研究进展寥寥。有例子表明对于PS和DS，期望反应模式和知识状态一一对应所需的条件迥然不同。这必将对其他相关研究产生"蝴蝶效应"，值得探究。对于PS-CD，本课题拟主要讨论：1.定义、设计和评估"好的"CD测验蓝图，即探讨PS-CD的Q矩阵理论；2.CD分类方法的开发和使用策略的探查；3.不同用途的测验宜选用何种方式及相应方式下如何选题、组卷；4.获得测验数据后如何评估测验质量、是否修改及如何修改Q矩阵。本课题是前三个基金项目（60263005, 30860084,31160203）的自然延伸。

多级评分数据比两级数据提供更多的信息，利用多级评分数据可以导出被试知识状态及加工策略的更准确估计。本课题研究多级评分和多值Q矩阵的Q矩阵理论，特别是相应的认知诊断测验设计：Q矩阵标定；Q矩阵理论应用研究；测验Q矩阵或认知诊断测验质量评价指标研究。.（1）给出了可达阵功能的不可替代性，认为Q矩阵中的列可以由可达阵（可达阵等价类中任何一个矩阵）的列表示出来；在非补偿，0-1评分条件下，测验Q矩阵可以使得知识状态和理想反应模式一一对应；这两种作用是Q矩阵中其他子矩阵不可代替的。.（2）给出了知识状态的两种表达形式：累赘表达式和简洁表达式；并且给出累赘表达式的初步应用，如给出不同的补救路径、简化关于可达阵在测验编制中的重要作用的定理的证明。.（3）给出完备Q矩阵、基本完备Q矩阵的概念，即包含列数最少而能够使知识状态和理论反应模式一一对应的测验Q矩阵；最后分别对这三种属性层级结构给出完备Q矩阵的构造，它们都由属性当中最大不可比较属性集合所表达的路径所决定，将基本属性层级结构依照离散数学中图论重新分类为根树型，菱形和独立型；并给出了给出最大不可比较属性集合的概念。.（4）对于属性之间无补偿，并且掌握题目中一个属性理想评分增加1分条件下，给出多级评分认知诊断测验的测验蓝图设计原理。丁树良等人给出多值Q矩阵理论。建立了多值Q矩阵和0-1 Q矩阵相互转化的方法，给出拟可达阵的概念和拟可达阵在多值Q矩阵测验设计中的作用。.（5）给出Q矩阵标定的方法；研究了认知诊断评估的信度和效度以及HCI指标；提出0-1评分认知诊断模型和多级评分多策略的诊断方法；给出具有认知诊断功能的计算机化测验（CD-CAT）的选题策略和终止规则；特别开发了具有认知诊断功能的多阶段计算机测验。.（6）培养了一批硕士和博士。发表了60多篇论文,撰写了一部认知诊断方面的专著，多次参加国际和国内的学术交流活动。.课题组成员同心协力进行研究，完成了课题既定任务。