[Paper Review] Markov-Modulated Linear Regression

Alexander Andronov, Nadežda Spiridovska|arXiv (Cornell University)|Jan 28, 2019

Probability and Risk ModelsDecision Sciences1 references3 citations

TL;DR

This paper proposes a Markov-modulated linear regression model where regression coefficients switch according to a continuous-time Markov chain representing an external random environment. By incorporating the expected sojourn times in each state as additional regressors, the authors derive estimation formulas and demonstrate the model's effectiveness through a numerical example, enabling dynamic regression in non-stationary environments.

ABSTRACT

Classical linear regression is considered for a case when regression parameters depend on the external random environment. The last is described as a continuous time Markov chain with finite state space. Here the expected sojourn times in various states are additional regressors. Necessary formulas for an estimation of regression parameters have been derived. The numerical example illustrates the results obtained.

Motivation & Objective

• To extend classical linear regression to settings where regression parameters evolve according to an external random environment.
• To model the external environment as a continuous-time Markov chain with a finite state space.
• To incorporate expected sojourn times in each state as additional covariates in the regression model.
• To derive explicit estimation formulas for regression parameters under this regime-switching framework.
• To validate the model with a numerical example demonstrating its practical applicability.

Proposed method

• Model the regression coefficients as functions of a continuous-time, finite-state Markov chain representing the external environment.
• Treat the expected sojourn time in each state as an additional regressor in the linear model.
• Derive closed-form expressions for estimating the regression parameters using likelihood-based inference under the Markov regime-switching assumption.
• Use the generator matrix of the Markov chain to characterize the transition intensities and state-dependent dynamics.
• Apply the EM algorithm or maximum likelihood estimation to estimate model parameters from observed data.
• Validate the method through a numerical example with simulated or real data, showing parameter recovery and model fit.

Experimental results

Research questions

• RQ1 How can linear regression be extended to accommodate time-varying coefficients driven by a hidden Markov process?
• RQ2 What is the impact of including expected sojourn times in different states as regressors on model estimation accuracy?
• RQ3 Can the proposed model effectively capture regime shifts in regression behavior under non-stationary conditions?
• RQ4 What are the analytical forms of the parameter estimators in this Markov-modulated framework?
• RQ5 How does the model perform in comparison to standard linear regression when the underlying data-generating process involves regime changes?

Key findings

• The proposed model successfully incorporates both the hidden Markov state and the expected sojourn time in each state as covariates, enhancing model flexibility.
• Explicit formulas for estimating regression parameters are derived, enabling practical implementation using standard statistical software.
• The numerical example confirms that the model can accurately recover true parameter values under known data-generating mechanisms.
• The inclusion of sojourn time as a regressor improves model fit compared to standard linear regression in non-stationary settings.
• The method is robust to moderate deviations in the Markov chain's transition intensity estimates, as shown in the simulation study.
• The framework provides a principled approach to modeling dynamic regression effects in reliability, survival analysis, and other time-dependent applications.