[Paper Review] Block stochastic gradient descent for large-scale tomographic reconstruction in a parallel network

Yushan Gao, Ander Biguri+1 more|arXiv (Cornell University)|Mar 28, 2019

Medical Imaging Techniques and Applications36 references3 citations

TL;DR

This paper proposes Block Stochastic Gradient Descent (BSGD), a parallel row-action method for large-scale tomographic reconstruction in distributed networks with limited node storage. By updating only blocks of the image per iteration using stochastic gradients and importance sampling, BSGD reduces inter-node communication while achieving faster convergence and improved accuracy over SIRT, CAV, and CSGD, especially with Gaussian noise and total variation regularization.

ABSTRACT

Iterative algorithms have many advantages for linear tomographic image reconstruction when compared to back-projection based methods. However, iterative methods tend to have significantly higher computational complexity. To overcome this, parallel processing schemes that can utilise several computing nodes are desirable. Popular methods here are row action methods, which update the entire image simultaneously and column action methods, which require access to all measurements at each node. In large scale tomographic reconstruction with limited storage capacity of each node, data communication overheads between nodes becomes a significant performance limiting factor. To reduce this overhead, we proposed a row action method BSGD. The method is based on the stochastic gradient descent method but it does not update the entire image at each iteration, which reduces between node communication. To further increase convergence speeds, an importance sampling strategy is proposed. We compare BSGD to other existing stochastic methods and show its effectiveness and efficiency. Other properties of BSGD are also explored, including its ability to incorporate total variation (TV) regularization and automatic parameter tuning.

Motivation & Objective

• Address the high computational complexity and communication overhead in large-scale iterative CT reconstruction using distributed computing.
• Reduce inter-node communication in parallel tomographic reconstruction by updating only image blocks instead of the full image at each iteration.
• Improve convergence speed and reconstruction quality in distributed settings with limited node storage capacity.
• Enable effective use of total variation regularization for sparse-view CT reconstruction in a distributed framework.
• Develop a method that combines stochastic optimization efficiency with scalability for large-scale, non-standard CT scan trajectories.

Proposed method

• Proposes BSGD as a block-wise stochastic gradient descent method that processes only subsets of projection data and image blocks per iteration, minimizing full-image updates.
• Uses importance sampling to prioritize more informative projection blocks, accelerating convergence.
• Employs a three-step iterative update: (1) residual tracking via auxiliary variable z, (2) gradient estimation via g, and (3) image update via x using step size μ.
• Introduces block-wise matrix partitioning of A into M row blocks and N column blocks, enabling distributed computation across nodes with partial data access.
• Incorporates total variation (TV) regularization by modifying the objective function and integrating it into the BSGD update rule.
• Uses randomized diagonal projection matrices R1, R2, R3 to model stochastic sampling and ensure convergence to the least-squares solution.

Experimental results

Research questions

• RQ1 Can a block-wise stochastic gradient method reduce communication overhead in distributed large-scale tomographic reconstruction without sacrificing convergence speed?
• RQ2 How does importance sampling improve convergence in BSGD compared to uniform sampling in stochastic gradient methods for CT?
• RQ3 To what extent does BSGD outperform established methods like SIRT, CAV, and CSGD in reconstruction accuracy and speed under Gaussian noise?
• RQ4 Can BSGD effectively integrate total variation regularization for sparse-view CT reconstruction in a distributed setting?
• RQ5 What is the theoretical convergence behavior of BSGD, and does it converge to the least-squares solution in the fixed-point analysis?

Key findings

• BSGD achieves faster convergence and higher reconstruction accuracy than SIRT, CAV, and CSGD in large-scale CT reconstruction with Gaussian noise.
• The use of importance sampling in BSGD significantly accelerates convergence compared to uniform sampling strategies.
• BSGD reduces inter-node communication by updating only image blocks per iteration, making it suitable for systems with limited node storage capacity.
• BSGD with total variation regularization (BSGD-TV) effectively reconstructs images from sparse-view data, demonstrating robustness in low-dose CT scenarios.
• Empirical results confirm convergence to the least-squares solution, and theoretical analysis suggests convergence to the optimal solution, though a formal proof is still pending.
• BSGD outperforms machine learning-based optimization methods like SVRG and ORBCDVD in computational efficiency for CT reconstruction tasks.