[Paper Review] TransientViT: A novel CNN - Vision Transformer hybrid real/bogus transient classifier for the Kilodegree Automatic Transient Survey

Zhuoyang Chen, Wenjie Zhou+4 more|arXiv (Cornell University)|Sep 18, 2023

CCD and CMOS Imaging SensorsEngineering3 citations

TL;DR

TransientViT is a novel CNN-Vision Transformer hybrid model that improves real/bogus transient classification in the Kilodegree Automatic Transient Survey (KATS) by combining local feature extraction via convolutional layers with global context modeling through hierarchical self-attention. It achieves a 99.44% accuracy and 0.97 AUC on the KATS-T 200K dataset, significantly reducing manual inspection needs in transient pipelines.

ABSTRACT

The detection and analysis of transient astronomical sources is of great importance to understand their time evolution. Traditional pipelines identify transient sources from difference (D) images derived by subtracting prior-observed reference images (R) from new science images (N), a process that involves extensive manual inspection. In this study, we present TransientViT, a hybrid convolutional neural network (CNN) - vision transformer (ViT) model to differentiate between transients and image artifacts for the Kilodegree Automatic Transient Survey (KATS). TransientViT utilizes CNNs to reduce the image resolution and a hierarchical attention mechanism to model features globally. We propose a novel KATS-T 200K dataset that combines the difference images with both long- and short-term images, providing a temporally continuous, multidimensional dataset. Using this dataset as the input, TransientViT achieved a superior performance in comparison to other transformer- and CNN-based models, with an overall area under the curve (AUC) of 0.97 and an accuracy of 99.44%. Ablation studies demonstrated the impact of different input channels, multi-input fusion methods, and cross-inference strategies on the model performance. As a final step, a voting-based ensemble to combine the inference results of three NRD images further improved the model's prediction reliability and robustness. This hybrid model will act as a crucial reference for future studies on real/bogus transient classification.

Motivation & Objective

• To address the high computational cost and limited global feature capture of pure CNN-based transient classifiers in time-domain astronomy.
• To improve the accuracy and robustness of real/bogus transient classification in the KATS pipeline, reducing reliance on manual inspection.
• To develop a temporally continuous, multidimensional dataset (KATS-T 200K) integrating long- and short-term difference images for enhanced model training.
• To evaluate the effectiveness of hybrid architectures combining CNNs and Vision Transformers for transient detection in noisy astronomical imaging.
• To optimize inference through multi-input fusion and cross-inference strategies, enhancing model reliability and generalization.

Proposed method

• The model uses convolutional layers to downsample input difference images and extract local spatial features.
• A hierarchical self-attention mechanism in the Vision Transformer component captures long-range dependencies and global context across the feature map.
• An adaptive cross-attention mechanism fuses features from multiple NRD (Near-Reference Difference) images, enhancing spatiotemporal representation learning.
• Multi-input fusion strategies—feature concatenation, element-wise addition, and adaptive cross-attention—were evaluated to optimize feature integration.
• A voting-based ensemble inference strategy combines predictions from three distinct NRD image segments to improve robustness and reliability.
• The model was trained and evaluated on the KATS-T 200K dataset, a novel, temporally continuous, multidimensional dataset combining difference images with long- and short-term temporal sequences.

Experimental results

Research questions

• RQ1 Can a hybrid CNN-ViT architecture outperform standalone CNN and ViT models in classifying real versus bogus transients in astronomical difference images?
• RQ2 How do different multi-input fusion techniques (concatenation, addition, adaptive cross-attention) affect the performance of transient classification models?
• RQ3 To what extent does cross-inference with unique sampling improve model generalization and prediction reliability?
• RQ4 What is the impact of input channel configuration (e.g., difference images only vs. multi-segment inputs) on model accuracy and AUC?
• RQ5 Can ensemble inference using multiple NRD segments significantly reduce false positive rates while preserving detection sensitivity?

Key findings

• TransientViT achieved a test accuracy of 99.44% and an AUC of 0.97 on the KATS-T 200K dataset, outperforming other ViT- and CNN-based models.
• The adaptive cross-attention fusion method yielded the highest performance among multi-input fusion strategies, achieving 98.66% accuracy and 97.87% AUC.
• Cross-inference with unique sampling improved model performance to 98.97% accuracy and 98.12% AUC, surpassing standard inference and random sampling strategies.
• The voting-based ensemble inference strategy further enhanced robustness, reducing the number of transient candidates per night by approximately 90% in the KATS pipeline.
• Ablation studies confirmed that input channel configuration and fusion method significantly influence model performance, with feature concatenation and adaptive cross-attention showing strong gains.
• Despite image defects (e.g., streaks from mount failures), the model maintained high performance, indicating resilience to common observational artifacts.