[Paper Review] Correspondence Analysis Using Neural Networks

Hsiang Hsu, Salman Salamatian+1 more|arXiv (Cornell University)|Feb 21, 2019

Sensory Analysis and Statistical Methods3 citations

TL;DR

This paper proposes CA-NN, a deep learning framework that reformulates correspondence analysis (CA) using principal inertia components (PICs) to enable scalable, high-dimensional CA on both discrete and continuous data. By framing CA as a functional optimization problem over neural networks, CA-NN efficiently approximates principal functions, outperforming traditional SVD-based CA on synthetic, recipe, wine, and image datasets with improved variance capture and interpretability.

ABSTRACT

Correspondence analysis (CA) is a multivariate statistical tool used to visualize and interpret data dependencies. CA has found applications in fields ranging from epidemiology to social sciences. However, current methods used to perform CA do not scale to large, high-dimensional datasets. By re-interpreting the objective in CA using an information-theoretic tool called the principal inertia components, we demonstrate that performing CA is equivalent to solving a functional optimization problem over the space of finite variance functions of two random variable. We show that this optimization problem, in turn, can be efficiently approximated by neural networks. The resulting formulation, called the correspondence analysis neural network (CA-NN), enables CA to be performed at an unprecedented scale. We validate the CA-NN on synthetic data, and demonstrate how it can be used to perform CA on a variety of datasets, including food recipes, wine compositions, and images. Our results outperform traditional methods used in CA, indicating that CA-NN can serve as a new, scalable tool for interpretability and visualization of complex dependencies between random variables.

Motivation & Objective

• To overcome the scalability and applicability limitations of traditional correspondence analysis (CA) on large, high-dimensional, or continuous datasets.
• To reframe CA as a functional optimization problem using principal inertia components (PICs), enabling a principled information-theoretic foundation.
• To develop a neural network-based method (CA-NN) that efficiently approximates the principal functions of PICs for both discrete and continuous random variables.
• To validate CA-NN on diverse real-world datasets, including food recipes, wine composition, and image data, demonstrating superior performance and interpretability.

Proposed method

• Reformulate CA as a functional optimization problem over finite-variance functions of two random variables using principal inertia components (PICs).
• Express the principal functions as solutions to a quadratic optimization problem derived from PIC theory.
• Approximate the optimal functions using multi-layer neural networks, enabling scalable learning on high-dimensional data.
• Train the CA-NN via end-to-end backpropagation to maximize the correlation between projected variables, effectively capturing dependency structure.
• Use encoder networks to map input features into low-dimensional representations that preserve statistical dependencies.
• Apply the learned principal functions to generate interpretable two-dimensional visualizations of variable relationships.

Experimental results

Research questions

• RQ1 Can correspondence analysis be reformulated as a functional optimization problem using principal inertia components to enable scalability to high-dimensional data?
• RQ2 How can neural networks effectively approximate the principal functions derived from PICs for both discrete and continuous random variables?
• RQ3 Does the proposed CA-NN method outperform traditional SVD-based CA in capturing variance and dependency structure in real-world datasets?
• RQ4 To what extent can CA-NN visualize meaningful clusters and interpretative patterns in complex data such as food recipes and wine compositions?
• RQ5 How does CA-NN generalize across diverse data types, including high-dimensional images and continuous features, where contingency tables are infeasible?

Key findings

• CA-NN successfully performs correspondence analysis on high-dimensional, continuous data such as MNIST and CIFAR-10, where traditional contingency table-based CA is infeasible.
• On the Kaggle What’s Cooking dataset, CA-NN revealed interpretable clusters separating East-Asian, Western, and Indian cuisines, with clear signature ingredients like curry and tortilla aligning with their respective cuisines.
• In the UCI Wine Quality dataset, CA-NN identified three sub-clusters of wine quality (poor, medium, high) and revealed that high-quality wines have lower citric and volatile acidity but higher sulphates.
• CA-NN captured higher correlation and variance in the first two principal functions compared to other deep learning-based CA methods, without requiring manual kernel selection.
• The linear interpolation path in the wine quality factorial plane showed a plausible trajectory from low- to high-quality wines, illustrating how feature changes influence perceived quality.
• CA-NN demonstrated robust performance on noisy MNIST and CIFAR-10, producing meaningful low-dimensional visualizations despite high input dimensionality and feature noise.