[Paper Review] A Comparative Analysis of Machine-learning Models for Solar Flare Forecasting: Identifying High-performing Active Region Flare Indicators

Suvadip Sinha, Om Gupta+10 more|arXiv (Cornell University)|Apr 12, 2022

Solar and Space Plasma Dynamics81 references33 citations

TL;DR

This study conducts a comprehensive comparative analysis of four machine learning models—k-nearest neighbors, logistic regression, random forest, and support vector machine—on HMI-derived magnetic features from Solar Cycle 24 to forecast solar flares. Logistic regression achieves the highest true skill score of 0.967 ± 0.018, with total current helicity, total vertical current density, and R VALUE emerging as top-performing flare indicators.

ABSTRACT

Solar flares create adverse space weather impacting space and Earth-based technologies. However, the difficulty of forecasting flares, and by extension severe space weather, is accentuated by the lack of any unique flare trigger or a single physical pathway. Studies indicate that multiple physical properties contribute to active region flare potential, compounding the challenge. Recent developments in machine learning (ML) have enabled analysis of higher-dimensional data leading to increasingly better flare forecasting techniques. However, consensus on high-performing flare predictors remains elusive. In the most comprehensive study to date, we conduct a comparative analysis of four popular ML techniques (k-nearest neighbor, logistic regression, random forest classifier, and support vector machine) by training these on magnetic parameters obtained from the Helioseismic and Magnetic Imager (HMI) on board the Solar Dynamics Observatory (SDO) for the entirety of solar cycle 24. We demonstrate that the logistic regression and support vector machine algorithms perform extremely well in forecasting active region flaring potential. The logistic regression algorithm returns the highest true skill score of $0.967 \pm 0.018$, possibly the highest classification performance achieved with any strictly parametric study. From a comparative assessment, we establish that the magnetic properties like total current helicity, total vertical current density, total unsigned flux, R_VALUE, and total absolute twist are the top-performing flare indicators. We also introduce and analyze two new performance metrics, namely, severe and clear space weather indicators. Our analysis constrains the most successful ML algorithms and identifies physical parameters that contribute most to active region flare productivity.

Motivation & Objective

• To evaluate and compare the performance of four popular machine learning models in forecasting solar flares using magnetic parameters.
• To identify the most predictive active region flare indicators from HMI SHARP data during Solar Cycle 24.
• To introduce and validate new performance metrics—severe and clear space weather indicators—for improved flare forecast evaluation.
• To reduce model dependency on feature count by identifying a minimal, low-correlation set of high-performing magnetic features.

Proposed method

• Trained four ML models—k-NN, logistic regression, random forest, and SVM—on 14 magnetic parameters derived from SDO/HMI SHARP data.
• Used a 24-hour flare forecast window and binary classification (flaring vs. non-flaring) on 1,376 active regions.
• Applied 10-fold cross-validation and computed performance using true skill score (TSS), precision, recall, and F1-score.
• Conducted feature importance ranking via permutation importance and correlation analysis to assess redundancy and distributional behavior.
• Introduced two new metrics: 'severe space weather indicator' (CSW) and 'clear space weather indicator' (CWS) to better evaluate non-flaring event prediction.
• Selected a minimal, low-correlation subset of features to test model robustness and performance stability.

Experimental results

Research questions

• RQ1 Which machine learning model performs best in forecasting solar flares using HMI magnetic parameters from Solar Cycle 24?
• RQ2 Which magnetic features are the most predictive of active region flare productivity?
• RQ3 How do the distributions of key features differ between flaring and non-flaring active regions, and how does this affect model performance?
• RQ4 To what extent can model performance be maintained with a reduced, low-correlation set of input features?
• RQ5 How do the new performance metrics (CSW and CWS) improve the evaluation of flare forecasting models compared to standard metrics?

Key findings

• Logistic regression achieved the highest true skill score of 0.967 ± 0.018, representing the best classification performance in any strictly parametric flare forecasting study to date.
• The top five flare indicators were total current helicity (TOTUSJH), total vertical current density (USFLUX), R VALUE, total unsigned flux (TOTUSJZ), and total absolute twist (TOTABSTWIST).
• Features like R VALUE showed a bimodal distribution in non-flaring regions, increasing variance and improving separation, which enhanced model discrimination.
• The model performance remained robust even with reduced feature sets, especially when internal correlation was minimized, indicating high efficiency in feature selection.
• The severe space weather indicator (CSW) metric highlighted that features like ABSNJZH, SAVNCPP, TOTFZ, and TOTPOT had improved ranking under CSW optimization due to sharp, low-variance distributions in the non-flaring class.
• The study confirms that high-performing models and key physical indicators can be systematically identified, enabling more precise and efficient operational flare forecasting.