[Paper Review] Repetitive Motion Estimation Network: Recover cardiac and respiratory signal from thoracic imaging

Xiaoxiao Li, Vivek Kumar Singh+4 more|arXiv (Cornell University)|Nov 8, 2018

Advanced MRI Techniques and Applications5 references3 citations

TL;DR

This paper proposes the Repetitive Motion Estimation Network (RMEN), a deep learning model that recovers cardiac and respiratory signals from thoracic fluoroscopic videos using only partial phase boundary annotations. By learning spatio-temporal repetition patterns and embedding them into 1D motion manifolds, RMEN reduces QRS peak detection offsets by 59.3% compared to baseline models, enabling accurate signal decomposition via frequency filtering even in irregular motion cases.

ABSTRACT

Tracking organ motion is important in image-guided interventions, but motion annotations are not always easily available. Thus, we propose Repetitive Motion Estimation Network (RMEN) to recover cardiac and respiratory signals. It learns the spatio-temporal repetition patterns, embedding high dimensional motion manifolds to 1D vectors with partial motion phase boundary annotations. Compared with the best alternative models, our proposed RMEN significantly decreased the QRS peaks detection offsets by 59.3%. Results showed that RMEN could handle the irregular cardiac and respiratory motion cases. Repetitive motion patterns learned by RMEN were visualized and indicated in the feature maps.

Motivation & Objective

• To recover cardiac and respiratory motion signals from thoracic fluoroscopic imaging without full motion annotations.
• To learn spatio-temporal repetition patterns in organ motion for motion manifold embedding.
• To enable accurate signal decomposition using frequency filtering after 1D embedding of motion patterns.
• To improve motion phase boundary detection accuracy in irregular motion scenarios such as skipped cycles or breathing holds.
• To provide a solution for image-guided interventions by estimating motion without external monitors or implanted markers.

Proposed method

• The model uses a 2D-Convolutional encoder with max pooling to extract spatial features from video frames.
• A two-layer stacked ConvLSTM processes the encoded temporal sequence to model motion dynamics.
• A 3D convolutional layer fuses features across channels for temporal context integration.
• Fully connected layers with dropout (0.5) regress the predicted motion phase from flattened features, using mean squared error loss.
• The network outputs a 1D phase curve using median prediction from multiple forward passes to reduce variance.
• Cardiac and respiratory signals are separated using zero-phase shift band-pass and low-pass filters based on physiological frequency ranges (0.5–2 Hz and 0.2–0.33 Hz, respectively).

Experimental results

Research questions

• RQ1 Can a deep learning model recover cardiac and respiratory motion signals from fluoroscopic videos using only partial phase boundary annotations?
• RQ2 Can the model learn and embed repetitive motion manifolds into a 1D signal for downstream signal decomposition?
• RQ3 How does the proposed RMEN model compare in accuracy to alternative methods like LSTM, SVR, and density-based approaches in QRS peak detection?
• RQ4 Can the model handle irregular motion patterns such as skipped cardiac cycles or breath-holding?
• RQ5 What kind of spatio-temporal patterns does the model learn, and are they interpretable in terms of anatomical motion?

Key findings

• RMEN reduced QRS peak detection offsets by 59.3% compared to the best alternative model, with an offset of 0.88 frames on the test set.
• The model achieved a true negative rate of 3 and 27 false positives on the test set, outperforming PCA+LSTM (9 and 72) and PCA+SVR (5 and 80).
• The DensityFlow unsupervised method had the highest offset (3.7 frames) and false positive rate (96), indicating inferior performance.
• Visualized feature maps from the Conv3D layer revealed clear repetitive patterns corresponding to cardiac (coronary) and respiratory (diaphragm) motion.
• The model successfully decomposed cardiac and respiratory signals using frequency filtering after 1D embedding, even in cases with motion irregularities.
• The method enables motion signal recovery without ECG monitors or implanted markers, supporting applications like dynamic coronary roadmapping.