[Paper Review] Neural Network Machine Regression (NNMR): A Deep Learning Framework for Uncovering High-order Synergistic Effects

Jiuchen Zhang, Ling Zhou+1 more|arXiv (Cornell University)|Feb 2, 2026

Nutritional Studies and Diet0 citations

TL;DR

NNMR integrates trainable input gating and adaptive depth regularization to jointly perform feature selection and nonlinear function estimation, with a post-selection permutation-based inference procedure to control type I error in high-dimensional settings.

ABSTRACT

We propose a new neural network framework, termed Neural Network Machine Regression (NNMR), which integrates trainable input gating and adaptive depth regularization to jointly perform feature selection and function estimation in an end-to-end manner. By penalizing both gating parameters and redundant layers, NNMR yields sparse and interpretable architectures while capturing complex nonlinear relationships driven by high-order synergistic effects. We further develop a post-selection inference procedure based on split-sample, permutation-based hypothesis testing, enabling valid inference without restrictive parametric assumptions. Compared with existing methods, including Bayesian kernel machine regression and widely used post hoc attribution techniques, NNMR scales efficiently to high-dimensional feature spaces while rigorously controlling type I error. Simulation studies demonstrate its superior selection accuracy and inference reliability. Finally, an empirical application reveals sparse, biologically meaningful food group predictors associated with somatic growth among adolescents living in Mexico City.

Motivation & Objective

• Motivate the need for interpretable, scalable feature selection in high-dimensional nonlinear settings and identify high-order synergistic effects.
• Propose an neural network framework that jointly selects features and estimates nonlinear functions via input gating and depth regularization.
• Develop a post-selection inference procedure combining data-splitting and permutation tests to ensure valid p-values after selection.

Proposed method

• Attach a trainable gating vector to the input layer to induce sparsity via L1 regularization on the gate (alpha).
• Introduce a depth-regularization penalty that collapses redundant layers to identity mappings, yielding a sparse, interpretable architecture.
• Train a neural network G(alpha, theta) with an end-to-end objective that combines squared error with L1 penalties on alpha and on (W_l - I) plus |c_l| (depth penalty).
• Perform post-selection inference using data splitting (D1 for selection/estimation, D2 for inference) and a permutation-based two-sample residual test to assess conditional independence.
• Provide truncation steps (hard-thresholding) to enforce exact zeros in alpha and to prune layers by setting W_l to I when small.

Experimental results

Research questions

• RQ1 Can NNMR accurately identify a sparse subset of predictive features in high-dimensional nonlinear settings?
• RQ2 Does integrating feature selection and depth regularization within a neural network improve selection accuracy and predictive performance relative to existing methods?
• RQ3 Can valid inference be performed after data-driven feature selection using a permutation-based two-sample residual test?
• RQ4 How does NNMR scale to high-dimensional data compared to BKMR and post hoc attribution methods?
• RQ5 Does the approach yield interpretable, biologically meaningful predictors in real data applications?

Key findings

• NNMR achieves superior variable selection metrics (Precision, Recall, F1) relative to BKMR, SHAP, DeepLIFT, GLNN, and DPS in simulations.
• Post-selection inference with the proposed permutation test maintains nominal Type I error after feature selection (1% vs 13% without selection).
• In simulations, NNMR’s F1 score is about 0.88, with precision 0.927 and recall 0.880 for the top features.
• Compared to BKMR, SHAP, and other baselines, NNMR shows stronger integration of sparsity and depth control during training, leading to better interpretability and scalability.
• Real data analysis (ELEMENT project) demonstrates NNMR’s ability to identify sparse, meaningful dietary predictors affecting infant growth, with AIC/time metrics reported for six methods (Table 3).