[Paper Review] Frequency Domain Convolutional Neural Network: Accelerated CNN for Large Diabetic Retinopathy Image Classification

Ee Fey Goh, Zhiyuan Chen+1 more|arXiv (Cornell University)|Jun 24, 2021

Retinal Imaging and Analysis15 references4 citations

TL;DR

This paper proposes Frequency Domain Convolutional Neural Networks (FDCNN) with Frequency Domain Convolution (FDC) and Frequency Domain Pooling (FDP) layers to accelerate training of large diabetic retinopathy (DR) images. By leveraging RFFT, kernel initialization, artifact removal, and Channel Independent Convolution, FDCNN achieves 54.21% faster training and 70.74% higher memory efficiency than standard CNNs, while the modified VGG16 with Full FDC layer reaches 95.63% accuracy on DR classification.

ABSTRACT

The conventional spatial convolution layers in the Convolutional Neural Networks (CNNs) are computationally expensive at the point where the training time could take days unless the number of layers, the number of training images or the size of the training images are reduced. The image size of 256x256 pixels is commonly used for most of the applications of CNN, but this image size is too small for applications like Diabetic Retinopathy (DR) classification where the image details are important for accurate classification. This research proposed Frequency Domain Convolution (FDC) and Frequency Domain Pooling (FDP) layers which were built with RFFT, kernel initialization strategy, convolution artifact removal and Channel Independent Convolution (CIC) to replace the conventional convolution and pooling layers. The FDC and FDP layers are used to build a Frequency Domain Convolutional Neural Network (FDCNN) to accelerate the training of large images for DR classification. The Full FDC layer is an extension of the FDC layer to allow direct use in conventional CNNs, it is also used to modify the VGG16 architecture. FDCNN is shown to be at least 54.21% faster and 70.74% more memory efficient compared to an equivalent CNN architecture. The modified VGG16 architecture with Full FDC layer is reported to achieve a shorter training time and a higher accuracy at 95.63% compared to the original VGG16 architecture for DR classification.

Motivation & Objective

• To address the high computational cost of training deep CNNs on large diabetic retinopathy (DR) images.
• To overcome the limitation of standard spatial convolution layers that are slow and memory-intensive, especially with high-resolution images.
• To develop frequency-domain alternatives—FDC and FDP layers—that maintain accuracy while improving training speed and efficiency.
• To enable effective use of large image sizes (e.g., 256x256 or larger) for DR classification, where fine-grained details are critical.
• To integrate the proposed layers into existing architectures like VGG16 to improve performance without architectural overhaul.

Proposed method

• Proposes Frequency Domain Convolution (FDC) layers using real-valued RFFT to transform spatial convolutions into frequency-domain operations, reducing computational complexity.
• Introduces a kernel initialization strategy tailored for frequency-domain learning to stabilize training and improve convergence.
• Applies convolution artifact removal techniques to minimize distortions introduced during frequency-domain operations.
• Employs Channel Independent Convolution (CIC) to decouple channel processing and enhance computational efficiency in the frequency domain.
• Develops Frequency Domain Pooling (FDP) layers that perform spatial downsampling in the frequency domain, preserving key features while reducing dimensionality.
• Introduces the Full FDC layer as a plug-in replacement for standard convolutional layers, enabling integration into existing CNNs like VGG16.

Experimental results

Research questions

• RQ1 Can frequency-domain operations significantly reduce training time and memory usage in CNNs for large medical images?
• RQ2 How does the performance of FDCNN compare to standard CNNs in terms of accuracy and efficiency on diabetic retinopathy classification?
• RQ3 To what extent can FDC and FDP layers preserve diagnostic-relevant features in high-resolution retinal images?
• RQ4 Can the Full FDC layer be effectively integrated into established architectures like VGG16 without retraining from scratch?
• RQ5 Does the proposed method maintain or improve classification accuracy while enabling the use of larger image inputs?

Key findings

• FDCNN achieves at least 54.21% faster training compared to an equivalent standard CNN architecture.
• The FDCNN model demonstrates 70.74% higher memory efficiency than conventional CNNs during training.
• The modified VGG16 architecture using the Full FDC layer reaches a test accuracy of 95.63% on diabetic retinopathy classification, outperforming the original VGG16.
• The use of frequency-domain operations enables effective training on larger image sizes without sacrificing speed or memory efficiency.
• The proposed kernel initialization and artifact removal techniques significantly improve training stability and feature fidelity in the frequency domain.
• The Full FDC layer enables seamless integration into existing CNNs, allowing performance gains without architectural redesign.