[Paper Review] Motion Pyramid Networks for Accurate and Efficient Cardiac Motion Estimation

Hanchao Yu, Xiaohong Chen+4 more|arXiv (Cornell University)|Jun 28, 2020

Advanced MRI Techniques and Applications28 references5 citations

TL;DR

This paper proposes Motion Pyramid Networks (MPN), a deep learning framework for accurate and efficient cardiac motion estimation in MRI. By fusing multi-scale motion fields and employing a cyclic teacher-student knowledge distillation strategy with progressive motion compensation, MPN achieves state-of-the-art performance with 26ms inference time per frame, significantly outperforming baseline models and deformable registration on public datasets.

ABSTRACT

Cardiac motion estimation plays a key role in MRI cardiac feature tracking and function assessment such as myocardium strain. In this paper, we propose Motion Pyramid Networks, a novel deep learning-based approach for accurate and efficient cardiac motion estimation. We predict and fuse a pyramid of motion fields from multiple scales of feature representations to generate a more refined motion field. We then use a novel cyclic teacher-student training strategy to make the inference end-to-end and further improve the tracking performance. Our teacher model provides more accurate motion estimation as supervision through progressive motion compensations. Our student model learns from the teacher model to estimate motion in a single step while maintaining accuracy. The teacher-student knowledge distillation is performed in a cyclic way for a further performance boost. Our proposed method outperforms a strong baseline model on two public available clinical datasets significantly, evaluated by a variety of metrics and the inference time. New evaluation metrics are also proposed to represent errors in a clinically meaningful manner.

Motivation & Objective

• To address the challenge of accurate and efficient cardiac motion estimation in cine MRI, particularly for clinical function assessment like myocardial strain.
• To overcome limitations of smoothness constraints and non-unique solutions in unsupervised deep learning-based motion estimation.
• To improve inference efficiency without sacrificing accuracy by enabling single-step motion estimation via knowledge distillation.
• To develop clinically meaningful evaluation metrics that decompose error into radial and circumferential directions for better alignment with myocardial strain assessment.

Proposed method

• The method uses a multi-scale feature pyramid from a shared encoder to generate motion fields at different resolutions, which are then fused to produce a refined motion field.
• A progressive motion compensation strategy is applied in the teacher model, iteratively refining motion estimates to handle large displacements and reduce reliance on smoothness constraints.
• A cyclic teacher-student distillation framework is introduced, where the student model learns to predict accurate motion in a single step by mimicking the teacher’s multi-step refinement.
• The student model is trained using the teacher’s output as supervision, enabling end-to-end training and faster inference while maintaining high accuracy.
• A novel motion pyramid architecture predicts motion independently at each scale before feature-level fusion, differing from autoregressive pyramid approaches.
• The framework is generalizable and applied to the ApparentFlow model, demonstrating effectiveness across different network architectures.

Experimental results

Research questions

• RQ1 Can multi-scale motion field fusion improve the accuracy and robustness of cardiac motion estimation in cine MRI?
• RQ2 How can knowledge distillation be leveraged to accelerate inference while preserving the performance of iterative motion compensation?
• RQ3 Can a cyclic training strategy with progressive refinement enhance motion estimation accuracy beyond standard distillation?
• RQ4 How do radial and circumferential motion errors differ in 2D cardiac tracking, and can error decomposition improve clinical relevance?
• RQ5 To what extent is the proposed method generalizable across different deep learning architectures for cardiac motion estimation?

Key findings

• MPN outperformed both a strong single-scale baseline and a conventional deformable registration method on two public datasets, achieving lower endpoint error and improved Dice scores.
• The cyclic knowledge distillation strategy (MPN-C) achieved the highest performance, with a 0.841 Dice score on the MYO structure in the ACDC dataset, surpassing the baseline ApparentFlow model.
• The proposed method reduced inference time to 26ms per frame, compared to 748ms for FFD-based registration, demonstrating significant efficiency gains.
• Error decomposition revealed that radial tracking was more accurate than circumferential tracking, highlighting the difficulty of tracking circular myocardial contours in 2D.
• The method generalized well to other models, as shown by improved performance on ApparentFlow when applying cyclic distillation and progressive compensation.
• The new evaluation metrics—error decomposition and key point tracking error—provided clinically meaningful insights into motion estimation accuracy beyond standard metrics.