[Paper Review] Pattern Dependence Detection using n-TARP Clustering

Tarun Yellamraju, Mireille Boutin|arXiv (Cornell University)|Jun 13, 2018

Neural Networks and Applications21 references3 citations

TL;DR

This paper proposes n-TARP clustering to detect statistically valid dependencies between high-dimensional observed variables and a one-dimensional outcome variable, especially in small-sample educational datasets. By projecting data onto random 1D subspaces, generating multiple binary clusterings, and validating them via permutation tests, the method quantifies linear and higher-order dependencies through empirical CDFs of outcome differences, revealing significant skill-grade relationships in a digital signal processing course with 27 students.

ABSTRACT

Consider an experiment involving a potentially small number of subjects. Some random variables are observed on each subject: a high-dimensional one called the "observed" random variable, and a one-dimensional one called the "outcome" random variable. We are interested in the dependencies between the observed random variable and the outcome random variable. We propose a method to quantify and validate the dependencies of the outcome random variable on the various patterns contained in the observed random variable. Different degrees of relationship are explored (linear, quadratic, cubic, ...). This work is motivated by the need to analyze educational data, which often involves high-dimensional data representing a small number of students. Thus our implementation is designed for a small number of subjects; however, it can be easily modified to handle a very large dataset. As an illustration, the proposed method is used to study the influence of certain skills on the course grade of students in a signal processing class. A valid dependency of the grade on the different skill patterns is observed in the data.

Motivation & Objective

• Address the challenge of detecting dependencies between high-dimensional observed variables and a one-dimensional outcome variable when sample sizes are small.
• Overcome ill-conditioning in regression due to high dimensionality and few samples by shifting focus from functional estimation to pattern-based clustering.
• Develop a scalable, computationally efficient method to identify statistically significant clusters of observed variable patterns that predict outcome variation.
• Quantify the strength and significance of dependencies, including higher-order (quadratic, cubic) relationships, and assess individual feature contributions.
• Illustrate the method’s utility in educational data analysis, particularly in identifying skill patterns influencing course grades with minimal subjects.

Proposed method

• Use random one-dimensional projections of high-dimensional observed variables to generate a large number of binary clusterings via a modified n-TARP classifier.
• Apply a statistical permutation test to each clustering to validate significance, discarding non-significant ones.
• Construct empirical cumulative distribution functions (CDFs) of the absolute difference in outcome variable means between cluster groups to quantify dependency strength.
• Compare the empirical CDFs against a null hypothesis distribution derived from Monte Carlo simulations to determine statistical significance levels.
• Extend the feature space by including polynomial terms (up to degree 3) to detect nonlinear dependencies between observed patterns and outcomes.
• Perform feature selection by sequentially removing features corresponding to individual skills and re-evaluating the CDFs to assess their influence on outcome dependency.

Experimental results

Research questions

• RQ1 Can pattern-based clustering detect statistically valid dependencies between high-dimensional student characteristics and course outcomes in small datasets?
• RQ2 How do linear, quadratic, and cubic relationships between observed variable patterns and outcomes compare in strength and significance?
• RQ3 Which individual skills (e.g., A, B, C, D, E) show significant influence on course grades, and at what order of dependency?
• RQ4 Can the method reliably produce distinct, statistically significant clusterings in high-dimensional, low-sample-size settings where traditional clustering fails?
• RQ5 How does the inclusion of higher-order polynomial features affect the detection of dependency patterns and the significance of outcome differences?

Key findings

• Out of 10,000 n-TARP clusterings, 20 were both distinct and statistically significant, demonstrating the method’s ability to generate reliable clusterings in a 26-dimensional, 27-sample dataset.
• The empirical CDF showed that 30% of statistically significant clusters had an average grade difference of at least 0.5, and this difference was within the two-sigma significance region.
• When feature space was extended to include quadratic terms, the significance region for outcome differences was largest, indicating stronger detectable dependencies at this order.
• For cubic terms, the CDF curve shifted upward when skill D was removed, indicating that skill D contributes significantly to grade dependency at degree three, though not at linear or quadratic levels.
• Linear dependency was primarily driven by skills A and E; removing these features caused the CDF to shift upward, indicating reduced dependency strength.
• The method outperformed standard clustering algorithms (e.g., Proclus, K-means) in this small-sample setting, as most failed to produce non-overlapping, statistically significant clusters.