[Paper Review] MedCoG: Maximizing LLM Inference Density in Medical Reasoning via Meta-Cognitive Regulation

Yu Zhao, Hao Guan+3 more|arXiv (Cornell University)|Feb 8, 2026

Artificial Intelligence in Healthcare and Education0 citations

TL;DR

MedCoG introduces a meta-cognition regulator that adaptively selects reasoning strategies (SCoT, memory, KG) for medical questions, achieving higher inference density and efficiency. It defines Inference Density and Inference Incremental Efficiency to quantify gains.

ABSTRACT

Large Language Models (LLMs) have shown strong potential in complex medical reasoning yet face diminishing gains under inference scaling laws. While existing studies augment LLMs with various knowledge types, it remains unclear how effectively the additional costs translate into accuracy. In this paper, we explore how meta-cognition of LLMs, i.e., their self-awareness of their own knowledge states, can regulate the reasoning process. Specifically, we propose MedCoG, a Medical Meta-Cognition Agent with Knowledge Graph, where the meta-cognitive assessments of task complexity, familiarity, and knowledge density dynamically regulate utilization of procedural, episodic, and factual knowledge. The LLM-centric on-demand reasoning aims to mitigate scaling laws by (1) reducing costs via avoiding indiscriminate scaling, (2) improving accuracy via filtering out distractive knowledge. To validate this, we empirically characterize the scaling curve and introduce inference density to quantify inference efficiency, defined as the ratio of theoretically effective cost to actual cost. Experiments demonstrate the effectiveness and efficiency of MedCoG on five hard sets of medical benchmarks, yielding 5.5x inference density. Furthermore, the Oracle study highlights the significant potential of meta-cognitive regulation.

Motivation & Objective

• Motivate why indiscriminate scaling of LLMs yields diminishing returns in medical reasoning.
• Propose a meta-cognitive regulator that self-assesses complexity, familiarity, and knowledge density to select reasoning strategies.
• Decouple procedural, episodic, and factual knowledge to enhance accuracy and efficiency.
• Introduce Knowledge Graph grounding with targeted verification to reduce noise and improve factual grounding.
• Quantify inference efficiency with Inference Density and Inference Incremental Efficiency (IIE).

Proposed method

• Propose MedCoG: a Medical Meta-Cognition Agent with Knowledge Graph consisting of a Meta-Cognition Regulator and an Executor.
• Regulator monitors three meta-cognitive dimensions (Complexity, Familiarity, Knowledge Density) and plans strategies via a threshold-based policy π(s) mapping to {Zero-Shot, SCoT, SCoT+Mem, SCoT+KG, SCoT+KG+Mem}.
• For factual knowledge, use KG verification targets and a constrained KG search space to avoid noise.
• Procedural knowledge is handled via SCoT; Episodic knowledge is retrieved from a memory case bank; Factual knowledge is grounded in a medical KG (PrimeKG) with entity grounding and path retrieval.
• Evaluation introduces Inference Density ρ and Inference Incremental Efficiency (IIE) to measure efficiency and cost-effectiveness.
• Experimental validation on five hard medical benchmarks shows MedCoG-Meta attaining higher inference density and IIE.

Experimental results

Research questions

• RQ1 Can meta-cognitive regulation improve the trade-off between accuracy and inference cost in medical reasoning?
• RQ2 How do complexity, familiarity, and knowledge density affect strategy selection and performance?
• RQ3 Does targeted integration of procedural, episodic, and factual knowledge outperform indiscriminate augmentation?
• RQ4 What are the upper bounds of Meta-Cognition regulation (Oracle study) and the practical gains with on-demand regulation?
• RQ5 How do new metrics (Inference Density, IIE) reflect improvements in inference efficiency?

Key findings

• MedCoG-Meta achieves 5.5× inference density over the Pareto frontier baseline.
• Oracle-style strategy selection within MedCoG approaches near-optimal performance, indicating strong upper-bound potential.
• Memory + KG synergy improves hard-case performance, while indiscriminate KG or memory alone can hurt performance.
• MedCoG-Meta yields the highest Inference Incremental Efficiency among baselines on hard medical benchmarks.
• Meta-cognition distributions vary by backbone model and dataset, showing domain-adaptive regulation benefits.