[Paper Review] Trimmed Constrained Mixed Effects Models: Formulations and Algorithms

Peng Zheng, Ryan M Barber+3 more|arXiv (Cornell University)|Sep 24, 2019

Statistical Methods and Bayesian InferenceMathematics36 references19 citations

TL;DR

This paper introduces LimeTr, an open-source Python package that extends robust least trimmed squares (LTS) to mixed effects models by incorporating trimming into marginal likelihood estimation. It enables robust, constrained, and nonlinear mixed effects modeling—particularly for meta-analysis and dose-response relationships—outperforming existing tools in outlier resistance and computational efficiency.

ABSTRACT

Mixed effects (ME) models inform a vast array of problems in the physical and social sciences, and are pervasive in meta-analysis. We consider ME models where the random effects component is linear. We then develop an efficient approach for a broad problem class that allows nonlinear measurements, priors, and constraints, and finds robust estimates in all of these cases using trimming in the associated marginal likelihood. The software accompanying this paper is disseminated as an open-source Python package called LimeTr. LimeTr is able to recover results more accurately in the presence of outliers compared to available packages for both standard longitudinal analysis and meta-analysis, and is also more computationally efficient than competing robust alternatives. Supplementary materials that reproduce the simulations, as well as run LimeTr and third party code are available online. We also present analyses of global health data, where we use advanced functionality of LimeTr, including constraints to impose monotonicity and concavity for dose-response relationships. Nonlinear observation models allow new analyses in place of classic approximations, such as log-linear models. Robust extensions in all analyses ensure that spurious data points do not drive our understanding of either mean relationships or between-study heterogeneity.

Motivation & Objective

• Address the lack of robust, constrained mixed effects models in meta-analysis and longitudinal studies, especially under outliers and nonlinear relationships.
• Develop a unified framework that combines trimming with linear mixed effects models to improve robustness against outliers in both fixed and random effects.
• Enable advanced modeling capabilities such as nonlinear observation models, shape constraints (e.g., monotonicity, concavity), and arbitrary nonlinear functions of fixed effects.
• Provide a computationally efficient and reliable optimization algorithm for nonconvex, nonsmooth problems arising from trimmed likelihoods.
• Demonstrate the method’s superiority on synthetic and real-world global health data, particularly in dose-response modeling with heterogeneous observations.

Proposed method

• Formulate a partially nonlinear mixed effects model where random effects enter linearly, but observation models and priors can be nonlinear.
• Introduce a trimming-based robust estimation approach by minimizing the sum of the smallest (1−α) squared residuals in the marginal likelihood, enhancing robustness to outliers.
• Reformulate the trimming problem using variable projection to decouple the optimization over trimming parameters and model parameters, enabling efficient solution via interior-point methods.
• Integrate constraints (linear and nonlinear) and shape constraints (e.g., monotonicity, concavity) into the optimization using spline-based modeling of nonlinear relationships.
• Implement the algorithm in LimeTr, a Python package that combines IPopt with custom solvers for the value function over trimming parameters.
• Use spline basis functions with shape constraints to model complex dose-response relationships, allowing flexible, interpretable, and robust inference.

Experimental results

Research questions

• RQ1 Can trimming be effectively extended to mixed effects models to improve robustness against outliers in both longitudinal and meta-analytic data?
• RQ2 How can nonlinear observation models and shape constraints (e.g., monotonicity) be incorporated into robust mixed effects estimation while maintaining computational efficiency?
• RQ3 Does the proposed trimmed constrained mixed effects model outperform existing robust packages in terms of accuracy and computational speed under contamination?
• RQ4 Can the method reliably recover true dose-response relationships from heterogeneous, indirect, or ratio-based observations in meta-analysis?
• RQ5 To what extent do shape constraints alter conclusions in real-world global health data, particularly when outliers or noisy data are present?

Key findings

• LimeTr outperforms existing packages (e.g., metafor, robumeta, robustlmm) in detecting and resisting the influence of outliers in heterogeneous longitudinal and meta-analytic data.
• The package achieves higher accuracy and computational efficiency than competing robust alternatives, particularly in the presence of contamination.
• LimeTr successfully models nonlinear dose-response relationships using splines with shape constraints, enabling robust inference without relying on log-linear approximations.
• In a meta-analysis of 14 prospective cohort studies on exposure and risk, LimeTr captured the cone of uncertainty in spline-based risk curves, with uncertainty quantified via both spline and random effects heterogeneity.
• Imposing monotonicity constraints on the dose-response curve altered the estimated risk relationship, demonstrating that shape constraints significantly affect conclusions—highlighting the importance of modeling flexibility.
• LimeTr is the only open-source package that supports nonlinear observation models, arbitrary constraints, and robust trimming simultaneously, enabling new types of analyses not feasible with prior tools.