[Paper Review] Contrastive Representation Learning for Predicting Solar Flares from Extremely Imbalanced Multivariate Time Series Data

Onur Vural, Shah Muhammad Hamdi+1 more|arXiv (Cornell University)|Oct 1, 2024

Market Dynamics and VolatilityEconomics, Econometrics and Finance3 citations

TL;DR

This paper proposes CONTREX, a novel contrastive representation learning framework for multivariate time series (MVTS) data to address extreme class imbalance in solar flare prediction. By extracting dynamic features, deriving extreme positive and negative class representations, and training a sequence embedding module with a custom contrastive reconstruction loss, CONTREX achieves state-of-the-art performance on the SWAN-SF benchmark, outperforming baselines in accuracy (0.7306), TSS (0.7098), and ROC AUC (0.8549).

ABSTRACT

Major solar flares are abrupt surges in the Sun's magnetic flux, presenting significant risks to technological infrastructure. In view of this, effectively predicting major flares from solar active region magnetic field data through machine learning methods becomes highly important in space weather research. Magnetic field data can be represented in multivariate time series modality where the data displays an extreme class imbalance due to the rarity of major flare events. In time series classification-based flare prediction, the use of contrastive representation learning methods has been relatively limited. In this paper, we introduce CONTREX, a novel contrastive representation learning approach for multivariate time series data, addressing challenges of temporal dependencies and extreme class imbalance. Our method involves extracting dynamic features from the multivariate time series instances, deriving two extremes from positive and negative class feature vectors that provide maximum separation capability, and training a sequence representation embedding module with the original multivariate time series data guided by our novel contrastive reconstruction loss to generate embeddings aligned with the extreme points. These embeddings capture essential time series characteristics and enhance discriminative power. Our approach shows promising solar flare prediction results on the Space Weather Analytics for Solar Flares (SWAN-SF) multivariate time series benchmark dataset against baseline methods.

Motivation & Objective

• Address the challenge of extreme class imbalance in solar flare prediction using multivariate time series (MVTS) data from solar active regions.
• Develop a contrastive representation learning framework tailored for temporal dependencies and rare event classification in MVTS.
• Improve discriminative power of learned embeddings by guiding them toward extreme class representations (positive and negative extremes).
• Demonstrate the effectiveness of the proposed method on the SWAN-SF benchmark dataset, a standard in solar flare prediction research.
• Enable better transferability and performance in downstream binary classification of major solar flares using contrastive learning principles.

Proposed method

• Extract dynamic features from each MVTS instance to capture essential time-series dynamics and temporal evolution.
• Compute two contrastive extreme points—one from positive (flaring) class and one from negative (non-flaring) class—maximizing inter-class separation.
• Train a sequence representation embedding module using the original MVTS data, guided by a novel contrastive reconstruction loss that aligns embeddings with the derived extreme points.
• Use the learned embeddings as input to a downstream logistic regression classifier for binary solar flare prediction.
• Design the contrastive reconstruction loss to encourage embeddings to be pulled toward the positive and negative extreme representations, enhancing discriminative capacity.
• Apply t-SNE visualization to validate the class separation capability of the learned embeddings in the latent space.

Figure 1: CONTREX is composed of (1) a sequence representation embedding module to derive fixed-dimensional embeddings from MVTS data points, and (2) a downstream classifier that utilizes the representation embeddings for binary prediction.

Experimental results

Research questions

• RQ1 Can contrastive representation learning improve performance in solar flare prediction under extreme class imbalance in multivariate time series?
• RQ2 How effective is the proposed contrastive reconstruction loss in guiding embeddings toward discriminative extreme class representations?
• RQ3 Does CONTREX outperform existing MVTS representation methods (e.g., LSTM, ROCKET, LTV) in terms of accuracy, TSS, and AUC on the SWAN-SF benchmark?
• RQ4 To what extent do the learned embeddings capture meaningful temporal and discriminative patterns for flare prediction?
• RQ5 Can the framework generalize to other MVTS datasets with binary or multi-class classification tasks?

Key findings

• CONTREX achieved the highest accuracy (0.7306 ± 0.0966) among all baselines on the SWAN-SF dataset, outperforming LTV by 5.1%.
• The model recorded the highest True Skill Statistic (TSS) of 0.7098 ± 0.0978, indicating strong performance in balancing sensitivity and specificity.
• CONTREX achieved the best ROC AUC score of 0.8549 ± 0.0489, demonstrating superior discrimination ability between flare and non-flare classes.
• Despite ranking second in F1 score (0.1579 ± 0.0477) and Heidke Skill Score (HSS2) (0.1189 ± 0.0497), CONTREX matched LTV in the GSS (0.02303 ± 0.0070), showing strong performance in balanced metrics.
• t-SNE visualization confirmed that CONTREX embeddings effectively separate positive and negative classes in the latent space, validating the model’s discriminative capability.
• The proposed contrastive reconstruction loss successfully guided the embedding space toward extreme class representations, enhancing feature discrimination.

Figure 2: Flare class distribution of each division in stacked bar plot format. For each partition, the count of five flare classes is shown in the plot.