Identifiability Conditions in Cognitive Diagnosis: Implications for Q-Matrix Estimation Algorithms (2024)

Kim, H., Köhn, H. F., & Chiu, C. Y. (2024). Identifiability Conditions in Cognitive Diagnosis: Implications for Q-Matrix Estimation Algorithms. In M. Wiberg, J.-S. Kim, H. Hwang, H. Wu, & T. Sweet (Eds.), Quantitative Psychology - The 88th Annual Meeting of the Psychometric Society, 2023 (pp. 25-35). (Springer Proceedings in Mathematics and Statistics; Vol. 452). Springer. https://doi.org/10.1007/978-3-031-55548-0_3

Identifiability Conditions in Cognitive Diagnosis: Implications for Q-Matrix Estimation Algorithms. / Kim, Hyunjoo; Köhn, Hans Friedrich; Chiu, Chia Yi.
Quantitative Psychology - The 88th Annual Meeting of the Psychometric Society, 2023. ed. / Marie Wiberg; Jee-Seon Kim; Heungsun Hwang; Hao Wu; Tracy Sweet. Springer, 2024. p. 25-35 (Springer Proceedings in Mathematics and Statistics; Vol. 452).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Kim, H, Köhn, HF & Chiu, CY 2024, Identifiability Conditions in Cognitive Diagnosis: Implications for Q-Matrix Estimation Algorithms. in M Wiberg, J-S Kim, H Hwang, H Wu & T Sweet (eds), Quantitative Psychology - The 88th Annual Meeting of the Psychometric Society, 2023. Springer Proceedings in Mathematics and Statistics, vol. 452, Springer, pp. 25-35, 88th Annual Meeting of the Psychometric Society, IMPS 2023, College Park, United States, 7/25/23. https://doi.org/10.1007/978-3-031-55548-0_3

Kim H, Köhn HF, Chiu CY. Identifiability Conditions in Cognitive Diagnosis: Implications for Q-Matrix Estimation Algorithms. In Wiberg M, Kim JS, Hwang H, Wu H, Sweet T, editors, Quantitative Psychology - The 88th Annual Meeting of the Psychometric Society, 2023. Springer. 2024. p. 25-35. (Springer Proceedings in Mathematics and Statistics). doi: 10.1007/978-3-031-55548-0_3

Kim, Hyunjoo ; Köhn, Hans Friedrich ; Chiu, Chia Yi. / Identifiability Conditions in Cognitive Diagnosis : Implications for Q-Matrix Estimation Algorithms. Quantitative Psychology - The 88th Annual Meeting of the Psychometric Society, 2023. editor / Marie Wiberg ; Jee-Seon Kim ; Heungsun Hwang ; Hao Wu ; Tracy Sweet. Springer, 2024. pp. 25-35 (Springer Proceedings in Mathematics and Statistics).

@inproceedings{6681e788c79a4bae8044e6d3e05e0a85,

title = "Identifiability Conditions in Cognitive Diagnosis: Implications for Q-Matrix Estimation Algorithms",

abstract = "The Q-matrix of a cognitive diagnosis (CD) assessment documents the item-attribute associations and is thus a key component of any CD test. However, the true Q-matrix underlying a CD assessment is never known; it must be estimated. In practice, this task is typically performed by content experts, which, however, can result in the misspecification of the Q-matrix, causing examinees to be misclassified. In response to these difficulties, algorithms have been developed for estimating the entire Q-matrix based on the item responses. Extant algorithms for estimating the Q-matrix under the conjunctive Deterministic Input Noisy “AND” Gate (DINA) model either impose the identifiability conditions from Chen et al. (J Amer Statist Assoc 110:850–866, 2015) or do not. The debate on which is “right” way to do is ongoing; especially, as these conditions are sufficient but not necessary, which means that viable alternative Q-matrix estimates may be ignored. The goal of this chapter was to compare the estimated Q-matrices obtained from three algorithms that do not impose the identifiability conditions on the Q-matrix estimator with the estimated Q-matrices obtained from two algorithms that do impose the identifiability conditions. Simulations were conducted using data conforming to the DINA model generated in using an identifiable “true” Q-matrix. The impact on Q-matrix estimation of three factors was controlled: the length of the test, the number of attributes, and the amount of error perturbation added to the data. The estimated Q-matrices were evaluated whether they met the identifiability conditions and in their capacity to enable the correct classification of examinees. The results show there is essentially no difference in the rates of correctly classified examinees between Q-matrix estimates obtained from algorithms imposing the identifiability conditions and those that do not.",

keywords = "Cognitive diagnosis, DINA model, Identifiability conditions, Q-matrix estimation",

author = "Hyunjoo Kim and K{\"o}hn, {Hans Friedrich} and Chiu, {Chia Yi}",

note = "Publisher Copyright: {\textcopyright} The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.; 88th Annual Meeting of the Psychometric Society, IMPS 2023 ; Conference date: 25-07-2023 Through 28-07-2023",

year = "2024",

doi = "10.1007/978-3-031-55548-0_3",

language = "English (US)",

isbn = "9783031555473",

series = "Springer Proceedings in Mathematics and Statistics",

publisher = "Springer",

pages = "25--35",

editor = "Marie Wiberg and Jee-Seon Kim and Heungsun Hwang and Hao Wu and Tracy Sweet",

booktitle = "Quantitative Psychology - The 88th Annual Meeting of the Psychometric Society, 2023",

address = "Germany",

}

TY - GEN

T1 - Identifiability Conditions in Cognitive Diagnosis

T2 - 88th Annual Meeting of the Psychometric Society, IMPS 2023

AU - Kim, Hyunjoo

AU - Köhn, Hans Friedrich

AU - Chiu, Chia Yi

N1 - Publisher Copyright:© The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.

PY - 2024

Y1 - 2024

N2 - The Q-matrix of a cognitive diagnosis (CD) assessment documents the item-attribute associations and is thus a key component of any CD test. However, the true Q-matrix underlying a CD assessment is never known; it must be estimated. In practice, this task is typically performed by content experts, which, however, can result in the misspecification of the Q-matrix, causing examinees to be misclassified. In response to these difficulties, algorithms have been developed for estimating the entire Q-matrix based on the item responses. Extant algorithms for estimating the Q-matrix under the conjunctive Deterministic Input Noisy “AND” Gate (DINA) model either impose the identifiability conditions from Chen et al. (J Amer Statist Assoc 110:850–866, 2015) or do not. The debate on which is “right” way to do is ongoing; especially, as these conditions are sufficient but not necessary, which means that viable alternative Q-matrix estimates may be ignored. The goal of this chapter was to compare the estimated Q-matrices obtained from three algorithms that do not impose the identifiability conditions on the Q-matrix estimator with the estimated Q-matrices obtained from two algorithms that do impose the identifiability conditions. Simulations were conducted using data conforming to the DINA model generated in using an identifiable “true” Q-matrix. The impact on Q-matrix estimation of three factors was controlled: the length of the test, the number of attributes, and the amount of error perturbation added to the data. The estimated Q-matrices were evaluated whether they met the identifiability conditions and in their capacity to enable the correct classification of examinees. The results show there is essentially no difference in the rates of correctly classified examinees between Q-matrix estimates obtained from algorithms imposing the identifiability conditions and those that do not.

AB - The Q-matrix of a cognitive diagnosis (CD) assessment documents the item-attribute associations and is thus a key component of any CD test. However, the true Q-matrix underlying a CD assessment is never known; it must be estimated. In practice, this task is typically performed by content experts, which, however, can result in the misspecification of the Q-matrix, causing examinees to be misclassified. In response to these difficulties, algorithms have been developed for estimating the entire Q-matrix based on the item responses. Extant algorithms for estimating the Q-matrix under the conjunctive Deterministic Input Noisy “AND” Gate (DINA) model either impose the identifiability conditions from Chen et al. (J Amer Statist Assoc 110:850–866, 2015) or do not. The debate on which is “right” way to do is ongoing; especially, as these conditions are sufficient but not necessary, which means that viable alternative Q-matrix estimates may be ignored. The goal of this chapter was to compare the estimated Q-matrices obtained from three algorithms that do not impose the identifiability conditions on the Q-matrix estimator with the estimated Q-matrices obtained from two algorithms that do impose the identifiability conditions. Simulations were conducted using data conforming to the DINA model generated in using an identifiable “true” Q-matrix. The impact on Q-matrix estimation of three factors was controlled: the length of the test, the number of attributes, and the amount of error perturbation added to the data. The estimated Q-matrices were evaluated whether they met the identifiability conditions and in their capacity to enable the correct classification of examinees. The results show there is essentially no difference in the rates of correctly classified examinees between Q-matrix estimates obtained from algorithms imposing the identifiability conditions and those that do not.

KW - Cognitive diagnosis

KW - DINA model

KW - Identifiability conditions

KW - Q-matrix estimation

UR - http://www.scopus.com/inward/record.url?scp=85199490562&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85199490562&partnerID=8YFLogxK

U2 - 10.1007/978-3-031-55548-0_3

DO - 10.1007/978-3-031-55548-0_3

M3 - Conference contribution

AN - SCOPUS:85199490562

SN - 9783031555473

T3 - Springer Proceedings in Mathematics and Statistics

SP - 25

EP - 35

BT - Quantitative Psychology - The 88th Annual Meeting of the Psychometric Society, 2023

A2 - Wiberg, Marie

A2 - Kim, Jee-Seon

A2 - Hwang, Heungsun

A2 - Wu, Hao

A2 - Sweet, Tracy

PB - Springer

Y2 - 25 July 2023 through 28 July 2023

ER -