[Paper Review] Evaluating the efficacy of sonification for signal detection in univariate, evenly sampled light curves using astronify

J. Tucker Brown, C. M. Harrison+2 more|arXiv (Cornell University)|Sep 9, 2022

Tactile and Sensory Interactions4 citations

TL;DR

This study evaluates the efficacy of sonification using the astronify tool for detecting transit-like signals in univariate, evenly sampled light curves. It finds that high signal-to-noise ratio (SNR ≥ 30) signals are reliably detected by both experts and non-experts via sonification, while medium SNR signals (7–10) require visual data for expert-level performance, suggesting sonification training is essential for broader utility in astronomical research.

ABSTRACT

Sonification is the technique of representing data with sound, with potential applications in astronomy research for aiding discovery and accessibility. Several astronomy-focused sonification tools have been developed; however, efficacy testing is extremely limited. We performed testing of astronify, a prototype tool for sonification functionality within the Barbara A. Mikulski Archive for Space Telescopes (MAST). We created synthetic light curves containing zero, one, or two transit-like signals with a range of signal-to-noise ratios (SNRs=3-100) and applied the default mapping of brightness to pitch. We performed remote testing, asking participants to count signals when presented with light curves as a sonification, visual plot, or combination of both. We obtained 192 responses, of which 118 self-classified as experts in astronomy and data analysis. For high SNRs (=30 and 100), experts and non-experts performed well with sonified data (85-100% successful signal counting). At low SNRs (=3 and 5) both groups were consistent with guessing with sonifications. At medium SNRs (=7 and 10), experts performed no better than non-experts with sonifications but significantly better (factor of ~2-3) with visuals. We infer that sonification training, like that experienced by experts for visual data inspection, will be important if this sonification method is to be useful for moderate SNR signal detection within astronomical archives and broader research. Nonetheless, we show that even a very simple, and non-optimised, sonification approach allows users to identify high SNR signals. A more optimised approach, for which we present ideas, would likely yield higher success for lower SNR signals.

Motivation & Objective

• To assess the effectiveness of sonification in detecting transit-like signals in astronomical light curves using the astronify tool.
• To investigate whether sonification can serve as a viable alternative or complement to visual data inspection for signal detection in astronomy.
• To determine whether expertise in astronomy and data analysis influences performance in sonification-based signal detection.
• To evaluate the impact of signal-to-noise ratio (SNR) on detection accuracy across different data representation formats (sonification, visual plots, combined).
• To explore the potential of sonification to enhance accessibility in astronomy research for blind and low-vision scientists.

Proposed method

• Generated synthetic univariate, evenly sampled light curves with zero, one, or two transit-like signals across a range of SNRs (3–100).
• Applied the default astronify sonification mapping—brightness to pitch—using the Python-based astronify tool.
• Presented participants with three data formats: sonified data only, visual light curve plots only, and combined visual+sonified representations.
• Conducted remote user testing with 192 participants, categorized as experts or non-experts in astronomy and data analysis.
• Collected responses to assess signal-counting accuracy across SNR levels and data formats.
• Performed statistical analysis to compare detection performance between expert and non-expert groups across different SNR regimes and data representations.

Experimental results

Research questions

• RQ1 Can sonification effectively detect transit-like signals in univariate, evenly sampled light curves at varying signal-to-noise ratios?
• RQ2 How does detection performance in sonification compare between experts and non-experts in astronomy and data analysis?
• RQ3 Does the combination of sonification and visual plots improve signal detection accuracy compared to either format alone?
• RQ4 At what SNR level does sonification become a reliable method for signal detection in astronomical data?
• RQ5 To what extent does prior expertise in data analysis enhance performance in sonification-based signal detection?

Key findings

• For high SNR signals (SNR = 30 and 100), both experts and non-experts achieved 85–100% success in correctly counting signals using sonified data.
• At low SNRs (3 and 5), detection performance across all groups and formats was consistent with random guessing, with success rates of 0–27%.
• For medium SNRs (7 and 10), experts achieved a success rate of 26% (+5/-4%) with sonified data only, but performed significantly better (factor of ~2.5) with visual or combined visual+sonified data.
• Non-experts showed similar performance (25% +10/-8%) with sonified data at medium SNRs but did not show a significant improvement with visual or combined formats.
• Experts were more accurate than non-experts at medium SNRs when using visual data, indicating that visual data inspection remains more effective for this group.
• The results suggest that sonification training or experience is likely necessary for sonification to match or exceed the performance of visual inspection, especially at medium SNR levels.