[Paper Review] Sufficient and Necessary Conditions for the Identifiability of the $Q$-matrix

Yuqi Gu, Gongjun Xu|arXiv (Cornell University)|Oct 9, 2018

Psychometric Methodologies and TestingDecision Sciences41 references16 citations

TL;DR

This paper establishes the first sufficient and necessary conditions for the joint identifiability of the $Q$-matrix and restricted latent class model (RLCM) parameters under the DINA model. It introduces three algebraic conditions—completeness, distinctness, and repetition—on the $Q$-matrix that guarantee identifiability and are easily verifiable in practice, resolving a long-standing open problem in cognitive diagnosis and providing minimal requirements for model estimability from data.

ABSTRACT

Restricted latent class models (RLCMs) have recently gained prominence in educational assessment, psychiatric evaluation, and medical diagnosis. Different from conventional latent class models, restrictions on the RLCM model parameters are imposed by a design matrix to respect practitioners' scientific assumptions. The design matrix, called $Q$-matrix in the cognitive diagnosis literature, is usually constructed by practitioners and domain experts, yet it is subjective and could be misspecified. To address this problem, researchers have proposed to estimate the $Q$-matrix from data. On the other hand, the fundamental learnability issue of the $Q$-matrix and model parameters remains underexplored and existing studies often impose stronger than needed or even impractical conditions. This paper proposes sufficient and necessary conditions for joint identifiability of the $Q$-matrix and the RLCM model parameters under different types of RLCMs. The developed identifiability conditions only depend on the design matrix and are easy to verify in practice.

Motivation & Objective

• To resolve the open problem of determining minimal, verifiable conditions for joint identifiability of the $Q$-matrix and RLCM parameters.
• To address the practical issue of $Q$-matrix misspecification in cognitive diagnosis, where expert-constructed matrices may be incorrect or unavailable.
• To provide a theoretical foundation for estimating both the $Q$-matrix and model parameters jointly from response data, ensuring valid inference.
• To extend identifiability results beyond the DINA model to general RLCMs by deriving necessary conditions that must hold for any such model.
• To introduce and characterize generic identifiability as a weaker, more practical alternative to strict identifiability in high-dimensional settings.

Proposed method

• Proposes three algebraic conditions on the $Q$-matrix: completeness (contains an identity submatrix), distinctness (all columns distinct except for identity submatrix parts), and repetition (each column has at least three 1s).
• Derives necessary and sufficient conditions for strict identifiability of the $Q$-matrix and DINA model parameters using these three properties.
• Applies duality between DINA and DINO models to extend identifiability results to the DINO model without additional proof.
• Introduces generic identifiability as a relaxation of strict identifiability, where non-identifiable parameter sets have Lebesgue measure zero.
• Uses numerical verification via MATLAB to compute marginal response probabilities and confirm indistinguishability of alternative parameter sets under different $Q$-matrices.
• Employs a systematic parameter construction method based on equation (S4.41) to generate 70 alternative parameter sets per example, all yielding identical response distributions.

Experimental results

Research questions

• RQ1 What are the minimal, verifiable conditions on the $Q$-matrix that ensure joint identifiability of the $Q$-matrix and DINA model parameters?
• RQ2 Can the identifiability conditions be expressed in a simple, algebraic form that is easy to check in practice?
• RQ3 How do the proposed conditions relate to existing sufficient conditions, and what is their necessity?
• RQ4 What is the role of generic identifiability in relaxing strict identifiability when the full conditions are too restrictive?
• RQ5 Can the identifiability framework be extended to general RLCMs beyond the DINA model?

Key findings

• The proposed conditions—completeness, distinctness, and repetition—are both necessary and sufficient for the joint identifiability of the $Q$-matrix and DINA model parameters.
• The identifiability conditions are expressed purely in terms of the $Q$-matrix structure, making them easy to verify without knowledge of model parameters.
• For $K=3, J=20$, a $Q$-matrix $Q_3$ was found to be non-identifiable, as 70 alternative parameter sets under a different $\bar{Q}_3$ produced identical response distributions with a maximum difference of $1.30 \times 10^{-18}$, below MATLAB machine precision.
• For $K=5, J=20$, a similar non-identifiability was confirmed for $Q_4$, with maximum response probability difference of $5.42 \times 10^{-19}$, again below machine error, confirming indistinguishability.
• The results imply that any $Q$-matrix failing to satisfy the three conditions cannot guarantee unique estimation of parameters, even with large sample sizes.
• Generic identifiability is established as a practical alternative, where non-identifiable parameter sets form a set of Lebesgue measure zero, allowing for reliable estimation in most cases despite theoretical non-identifiability.