[Paper Review] CTformer: Convolution-free Token2Token Dilated Vision Transformer for Low-dose CT Denoising

Dayang Wang, Fenglei Fan+4 more|arXiv (Cornell University)|Feb 28, 2022

Medical Imaging Techniques and Applications5 citations

TL;DR

This paper proposes CTformer, a convolution-free, token-to-token dilated vision transformer for low-dose CT denoising that replaces convolutions with adaptive token rearrangement and dilated feature maps to enhance long-range context capture. It achieves state-of-the-art performance with lower computation and improved interpretability through attention visualization and dynamic flow tracing, while an overlapped inference mechanism reduces boundary artifacts.

ABSTRACT

Low-dose computed tomography (LDCT) denoising is an important problem in CT research. Compared to the normal dose CT (NDCT), LDCT images are subjected to severe noise and artifacts. Recently in many studies, vision transformers have shown superior feature representation ability over convolutional neural networks (CNNs). However, unlike CNNs, the potential of vision transformers in LDCT denoising was little explored so far. To fill this gap, we propose a Convolution-free Token2Token Dilated Vision Transformer for low-dose CT denoising. The CTformer uses a more powerful token rearrangement to encompass local contextual information and thus avoids convolution. It also dilates and shifts feature maps to capture longer-range interaction. We interpret the CTformer by statically inspecting patterns of its internal attention maps and dynamically tracing the hierarchical attention flow with an explanatory graph. Furthermore, an overlapped inference mechanism is introduced to effectively eliminate the boundary artifacts that are common for encoder-decoder-based denoising models. Experimental results on Mayo LDCT dataset suggest that the CTformer outperforms the state-of-the-art denoising methods with a low computation overhead.

Motivation & Objective

• To address the challenge of severe noise and artifacts in low-dose CT (LDCT) images, which limit clinical utility.
• To explore the potential of pure vision transformers in LDCT denoising, overcoming limitations of CNNs in capturing long-range dependencies.
• To develop a model that is not only effective but also interpretable for clinical trust, using attention visualization and dynamic flow tracing.
• To eliminate boundary artifacts common in encoder-decoder architectures through an overlapped inference mechanism.
• To demonstrate that a pure transformer approach can outperform hybrid CNN-transformer models in LDCT denoising with lower computational cost.

Proposed method

• The CTformer replaces convolutional operations with a Token2Token Dilated (T2TD) block that reorganizes tokens across spatial regions to enhance local and global feature integration without convolutions.
• It employs dilated and shifted feature maps to expand the receptive field and capture long-range dependencies, improving contextual representation.
• The model uses a hierarchical self-attention mechanism with cyclic shift to enable dynamic, multi-scale feature learning across layers.
• An overlapped inference strategy is introduced to mitigate boundary artifacts by allowing overlapping receptive fields during testing.
• Interpretability is achieved through static attention map analysis and dynamic attention flow tracing using TopK and Local Maximum (LM) graphs to visualize activation propagation.
• The architecture is trained end-to-end on the Mayo LDCT dataset using a reconstruction loss to map noisy LDCT images to their corresponding normal-dose CT (NDCT) counterparts.

Experimental results

Research questions

• RQ1 Can a pure vision transformer architecture outperform CNN-based and hybrid models in low-dose CT denoising without using convolutional operations?
• RQ2 How does the proposed T2TD block with dilated and shifted feature maps improve feature representation compared to standard ViT or Swin Transformer designs?
• RQ3 To what extent can self-attention maps in the CTformer be interpreted to reveal the model’s internal decision-making process for clinical trust?
• RQ4 Does the proposed overlapped inference mechanism effectively reduce boundary artifacts common in autoencoder-style denoising models?
• RQ5 What is the optimal depth (number of transformer blocks) for the CTformer in terms of performance and training efficiency?

Key findings

• The CTformer achieves a SSIM of 0.9121 and RMSE of 9.0233 on the Mayo LDCT dataset, outperforming the state-of-the-art method Solve-ViT by 0.0235 in SSIM and 3.3362 in RMSE.
• The model with only one transformer block achieves the best performance, indicating that deeper networks do not improve denoising and increase training time significantly.
• The T2TD block improves feature integration, reducing blotchy artifacts seen in the Sole-ViT baseline, which uses fixed-region tokenization and a single convolution.
• The cyclic shift operation improves SSIM by 0.0026 and reduces RMSE by 0.1337 compared to the CTformer without cyclic shift.
• Attention visualization reveals coherent dynamic flow: local maxima activations concentrate on the same spatial points across layers, indicating consistent co-activation of structural components during noise reduction.
• The proposed interpretation framework—using TopK and LM graphs—successfully traces hierarchical attention flow, enabling visual and dynamic understanding of the model’s latent behavior.