[Paper Review] Hybrid Retrieval-Generation Reinforced Agent for Medical Image Report Generation

Christy Y. Li, Xiaodan Liang+2 more|arXiv (Cornell University)|May 21, 2018

Multimodal Machine Learning ApplicationsComputer Science45 references151 citations

TL;DR

HRGR-Agent combines retrieval of template sentences with generation-based wording, trained via hierarchical reinforcement learning, to produce structured, accurate medical reports and achieve state-of-the-art results on two chest X-ray datasets.

ABSTRACT

Generating long and coherent reports to describe medical images poses challenges to bridging visual patterns with informative human linguistic descriptions. We propose a novel Hybrid Retrieval-Generation Reinforced Agent (HRGR-Agent) which reconciles traditional retrieval-based approaches populated with human prior knowledge, with modern learning-based approaches to achieve structured, robust, and diverse report generation. HRGR-Agent employs a hierarchical decision-making procedure. For each sentence, a high-level retrieval policy module chooses to either retrieve a template sentence from an off-the-shelf template database, or invoke a low-level generation module to generate a new sentence. HRGR-Agent is updated via reinforcement learning, guided by sentence-level and word-level rewards. Experiments show that our approach achieves the state-of-the-art results on two medical report datasets, generating well-balanced structured sentences with robust coverage of heterogeneous medical report contents. In addition, our model achieves the highest detection accuracy of medical terminologies, and improved human evaluation performance.

Motivation & Objective

• Motivation to improve long, coherent medical image reports by balancing template-based reliability with flexible generation.
• Leverage human prior knowledge via a template database to guide report structure and terminology.
• Develop a hierarchical decision process that selects between retrieval and generation per sentence.
• Train the retrieval and generation components jointly using reinforcement learning with sentence-level and word-level rewards.

Proposed method

• Represent a medical image as a context vector from a CNN-based image encoder and a sentence-topic sequence q generated by a sentence decoder with attention.
• Use a retrieval policy module to decide, for each topic, between retrieving a template sentence from a database or generating a new sentence.
• Maintain a template database built from frequent sentences in training reports (grouped by meaning).
• Implement a generation module that produces sentences conditioned on the topic and image context when generation is chosen.
• Train with hierarchical reinforcement learning: sentence-level rewards (for retrieval policy) and word-level rewards (for generation), using CIDEr-based rewards and a delta-CIDEr formulation for both levels.

Experimental results

Research questions

• RQ1 How can retrieval of templates and neural generation be integrated to improve long medical reports without sacrificing accuracy or diversity?
• RQ2 Does hierarchical reinforcement learning improve long-term coherence and factual grounding in medical report generation compared to pure generation or pure retrieval baselines?
• RQ3 Can a template-based retrieval component enhance the coverage of abnormal findings while maintaining linguistic quality?
• RQ4 What are the impacts of end-to-end RL training on both the retrieval policy and the generation module in medical reporting?

Key findings

• HRGR-Agent achieves state-of-the-art CIDEr/BLEU/ROUGE on two chest X-ray report datasets compared to non-retrieval baselines.
• The model balances template-like sentences and generated content, improving coverage of abnormal findings while maintaining fluent language.
• Retrieval alone yields strong automatic scores but weaker medical abnormality detection; combining retrieval with generation via RL yields higher precision in abnormal terminology.
• HRGR-Agent provides better human evaluation preference scores than baselines, indicating more natural, accurate reports with appropriate terminology.
• In CX-CHR, the HRGR-Agent outperforms the generation-only variant (HRG) by CIDEr (2.895 vs 2.800) and BLEU/ROUGE metrics, demonstrating the benefit of the retrieval-augmented approach.
• On IU X-Ray, HRGR-Agent achieves higher CIDEr and BLEU/ROUGE than several baselines, with notable improvements over purely retrieval or purely generation approaches.