[Paper Review] Integrating Random Forests and Generalized Linear Models for Improved Accuracy and Interpretability

Abhineet Agarwal, Ana Kenney+3 more|arXiv (Cornell University)|Jul 4, 2023

Gene expression and cancer classification8 citations

TL;DR

The provided text is a template with placeholder content; no substantive methods, results, or research questions are described.

ABSTRACT

Random forests (RFs) are among the most popular supervised learning algorithms due to their nonlinear flexibility and ease-of-use. However, as black box models, they can only be interpreted via algorithmically-defined feature importance methods, such as Mean Decrease in Impurity (MDI), which have been observed to be highly unstable and have ambiguous scientific meaning. Furthermore, they can perform poorly in the presence of smooth or additive structure. To address this, we reinterpret decision trees and MDI as linear regression and $R^2$ values, respectively, with respect to engineered features associated with the tree's decision splits. This allows us to combine the respective strengths of RFs and generalized linear models in a framework called RF+, which also yields an improved feature importance method we call MDI+. Through extensive data-inspired simulations and real-world datasets, we show that RF+ improves prediction accuracy over RFs and that MDI+ outperforms popular feature importance measures in identifying signal features, often yielding more than a 10% improvement over its closest competitor. In case studies on drug response prediction and breast cancer subtyping, we further show that MDI+ extracts well-established genes with significantly greater stability compared to existing feature importance measures.

Motivation & Objective

• The document appears to be a formatting/template sample rather than reporting actual research objectives.
• No concrete motivation or scientific goal is described in the provided text.
• The abstract, methods, and verification sections are placeholders without real content.

Proposed method

• No actual methodological content is provided in the text; sections are labeled as placeholders.
• The document demonstrates formatting guidelines (e.g., margins, figures, tables) rather than presenting a new methodological contribution.
• Key techniques or equations are not described in the supplied material.

Figure 1: Consistency comparison in fitting surrogate model in the tidal power example.

Experimental results

Research questions

• RQ1 No explicit research questions are present in the provided text.

Key findings

• No empirical results or quantitative findings are reported in the supplied content.