[Paper Review] Prediction of Solar Proton Events with Machine Learning: Comparison with Operational Forecasts and "All-Clear" Perspectives

Viacheslav M. Sadykov, А. Г. Косовичев+8 more|arXiv (Cornell University)|Jul 8, 2021

Solar and Space Plasma Dynamics2 references4 citations

TL;DR

This paper proposes a machine learning model using a custom artificial neural network to predict solar proton events (SPEs) based on solar active region magnetic fields, preceding soft X-ray and proton fluxes, and radio burst statistics. The model outperforms NOAA SWPC operational forecasts—especially in minimizing missed events—demonstrating the feasibility of robust 'all-clear' SPE predictions using ML.

ABSTRACT

Solar Energetic Particle events (SEPs) are among the most dangerous transient phenomena of solar activity. As hazardous radiation, SEPs may affect the health of astronauts in outer space and adversely impact current and future space exploration. In this paper, we consider the problem of daily prediction of Solar Proton Events (SPEs) based on the characteristics of the magnetic fields in solar Active Regions (ARs), preceding soft X-ray and proton fluxes, and statistics of solar radio bursts. The machine learning (ML) algorithm uses an artificial neural network of custom architecture designed for whole-Sun input. The predictions of the ML model are compared with the SWPC NOAA operational forecasts of SPEs. Our preliminary results indicate that 1) for the AR-based predictions, it is necessary to take into account ARs at the western limb and on the far side of the Sun; 2) characteristics of the preceding proton flux represent the most valuable input for prediction; 3) daily median characteristics of ARs and the counts of type II, III, and IV radio bursts may be excluded from the forecast without performance loss; and 4) ML-based forecasts outperform SWPC NOAA forecasts in situations in which missing SPE events is very undesirable. The introduced approach indicates the possibility of developing robust "all-clear" SPE forecasts by employing machine learning methods.

Motivation & Objective

• To develop a machine learning-based daily forecast system for solar proton events (SPEs) targeting high-sensitivity detection to minimize missed events.
• To compare the performance of the ML model against operational NOAA SWPC forecasts using real-time and definitive data.
• To identify the most predictive input features for SPE forecasting, including AR characteristics, proton flux, and radio burst counts.
• To explore the feasibility of 'all-clear' SPE forecasts using only GOES observations, reducing reliance on complex AR data.
• To assess the impact of data scarcity and model setup on forecast reliability, particularly given limited SPE events in Solar Cycle 24.

Proposed method

• A custom artificial neural network architecture is trained on whole-Sun input data, integrating characteristics from solar active regions (ARs), including those on the western limb and far side.
• Input features include preceding 10 MeV proton flux, soft X-ray (SXR) flux from flares, and counts of type II, III, and IV radio bursts.
• The model is trained and tested on definitive SHARP CEA data from Solar Cycle 24, with no separate validation set due to data scarcity.
• Performance is evaluated using the weighted true skill statistic (WTSS) and receiver operating characteristic (ROC) curves.
• Model comparisons are made directly with NOAA SWPC operational forecasts using the same test set, ensuring fair evaluation.
• The model is evaluated under class-imbalanced conditions, with a focus on minimizing false negatives in SPE prediction.

Experimental results

Research questions

• RQ1 Can a machine learning model outperform NOAA SWPC operational forecasts in predicting solar proton events, especially in minimizing missed events?
• RQ2 Which input features—proton flux, SXR flux, AR characteristics, or radio burst counts—are most predictive for SPE forecasting?
• RQ3 To what extent can SPE forecasts be simplified by excluding AR and radio burst data without performance loss?
• RQ4 How does the inclusion of ARs on the western limb and far side of the Sun affect prediction accuracy?
• RQ5 Can a robust 'all-clear' SPE forecast be developed using only GOES-based observations, enabling real-time operational use?

Key findings

• The ML-based forecast outperforms NOAA SWPC forecasts for all α > 1 in the WTSS metric, indicating superior performance in minimizing false negatives.
• Including proxies for ARs on the western limb and far side significantly improves prediction accuracy when using AR-based inputs.
• Preceding 10 MeV proton flux is the most valuable input feature for SPE prediction, followed by soft X-ray flux characteristics.
• Excluding AR characteristics and radio burst counts does not degrade performance and may even improve it, enabling a simplified forecast based solely on GOES observations.
• The model demonstrates strong potential for 'all-clear' SPE forecasting, where missing an event is highly undesirable, due to its low false negative rate.
• Despite data scarcity (only 101 SPE days in Solar Cycle 24), the model achieves reliable performance, suggesting feasibility for broader application with extended data sets.