[Paper Review] Efficient Complex-Valued Vision Transformers for MRI Classification Directly from k-Space

Moritz Rempe, Lukas T. Rotkopf+7 more|arXiv (Cornell University)|Jan 26, 2026

Advanced MRI Techniques and Applications0 citations

TL;DR

Introduces kViT, a fully complex-valued Vision Transformer that operates directly on undersampled k-Space MRI data with radial patches and complex positional embeddings, achieving competitive accuracy while drastically reducing VRAM usage.

ABSTRACT

Deep learning applications in Magnetic Resonance Imaging (MRI) predominantly operate on reconstructed magnitude images, a process that discards phase information and requires computationally expensive transforms. Standard neural network architectures rely on local operations (convolutions or grid-patches) that are ill-suited for the global, non-local nature of raw frequency-domain (k-Space) data. In this work, we propose a novel complex-valued Vision Transformer (kViT) designed to perform classification directly on k-Space data. To bridge the geometric disconnect between current architectures and MRI physics, we introduce a radial k-Space patching strategy that respects the spectral energy distribution of the frequency-domain. Extensive experiments on the fastMRI and in-house datasets demonstrate that our approach achieves classification performance competitive with state-of-the-art image-domain baselines (ResNet, EfficientNet, ViT). Crucially, kViT exhibits superior robustness to high acceleration factors and offers a paradigm shift in computational efficiency, reducing VRAM consumption during training by up to 68$ imes$ compared to standard methods. This establishes a pathway for resource-efficient, direct-from-scanner AI analysis.

Motivation & Objective

• Motivate AI analysis directly on MRI k-Space data to preserve phase information and avoid lossy reconstructions.
• Develop a complex-valued ViT architecture suited for non-local k-Space data.
• Propose a physics-informed radial k-Space patching strategy and complex positional embeddings.
• Evaluate robustness to undersampling across multiple MRI datasets.
• Demonstrate VRAM efficiency advantages over standard image-domain baselines.

Proposed method

• Implement a fully complex-valued transformer with complex-valued multi-head self-attention and feed-forward networks.
• Introduce radial k-Space patching that segments pixels by radial distance from k-Space center.
• Apply complex-valued linear projection to patches and use complex-valued positional embeddings (learnable or RoPE-based).
• Train on fastMRI Prostate and Knee, and in-house Glioma datasets with undersampling factors up to 24x.
• Compare against real-valued ViT, EfficientNet, and ResNet baselines.
• Evaluate with AUROC and AUPRC across 5-fold cross-validation and various acceleration factors.

Experimental results

Research questions

• RQ1 Can a complex-valued Vision Transformer operating directly on k-Space data match or exceed image-domain baselines for MRI classification?
• RQ2 Does radial patching better capture k-Space structure than Cartesian patches in a ViT for MRI tasks?
• RQ3 How does k-SViT performance and resource usage scale with undersampling compared to real-valued models?
• RQ4 What is the impact of complex positional embeddings on performance in k-Space transformers?
• RQ5 Is the approach robust across different MRI modalities and pathology tasks (prostate, knee, brain glioma)?

Key findings

• kViT achieves competitive AUROC and AUPRC with image-domain baselines while dramatically reducing VRAM usage during training (up to 68× in MIL setting).
• On fastMRI Prostate, kViT maintains robust performance at high undersampling (16×) with VRAM far lower than ResNet50.
• On fastMRI Knee, kViT reaches comparable performance to SOTA with reduced VRAM, though performance declines at higher undersampling similar to baselines.
• MIL experiments show kViT with substantially lower VRAM (0.52–0.96 GB) yet strong patient-level AUPRC and competitive AUROC.
• Attention maps indicate k-Space center focus with attention to outer high-frequency regions, aligning with spectral information use.
• Ablations show 16 rings as optimal for patching, highlight importance of phase information, and demonstrate benefits of Cutout augmentation.