[Paper Review] From Performance to Practice: Knowledge-Distilled Segmentator for On-Premises Clinical Workflows

Qizhen Lan, Aaron Choi+5 more|arXiv (Cornell University)|Jan 14, 2026

Scientific Computing and Data Management0 citations

TL;DR

The paper presents a logit-based knowledge distillation framework to compress a high-capacity nnU-Net segmentation model into a family of compact student models, enabling deployment on resource-constrained on-premises clinical workflows while preserving accuracy and improving efficiency.

ABSTRACT

Deploying medical image segmentation models in routine clinical workflows is often constrained by on-premises infrastructure, where computational resources are fixed and cloud-based inference may be restricted by governance and security policies. While high-capacity models achieve strong segmentation accuracy, their computational demands hinder practical deployment and long-term maintainability in hospital environments. We present a deployment-oriented framework that leverages knowledge distillation to translate a high-performing segmentation model into a scalable family of compact student models, without modifying the inference pipeline. The proposed approach preserves architectural compatibility with existing clinical systems while enabling systematic capacity reduction. The framework is evaluated on a multi-site brain MRI dataset comprising 1,104 3D volumes, with independent testing on 101 curated cases, and is further examined on abdominal CT to assess cross-modality generalizability. Under aggressive parameter reduction (94%), the distilled student model preserves nearly all of the teacher's segmentation accuracy (98.7%), while achieving substantial efficiency gains, including up to a 67% reduction in CPU inference latency without additional deployment overhead. These results demonstrate that knowledge distillation provides a practical and reliable pathway for converting research-grade segmentation models into maintainable, deployment-ready components for on-premises clinical workflows in real-world health systems.

Motivation & Objective

• Bridge the gap between research-grade segmentation performance and on-premises clinical deployment constraints.
• Develop a unified logit-based knowledge distillation framework to compress a high-capacity teacher into deployable student models.
• Preserve architectural compatibility with existing clinical pipelines while enabling scalable channel-reduced students.
• Demonstrate cross-modality generalizability (MRI brain and abdominal CT) under fixed deployment pipelines.
• Show that KD improves stability and boundary fidelity under aggressive compression relevant to health-system workflows.

Proposed method

• Use a high-capacity nnU-Net teacher trained on multi-site brain MRI to provide soft targets for distillation.
• Define a scalable family of student models by uniformly reducing channel width to 1/2 and 1/4, preserving the same architecture and deployment interface.
• Train students with a combined loss of standard segmentation loss and KL-divergence distillation loss on softened teacher outputs with temperature tau.
• Optimize for deployment by keeping inference pipeline unchanged while training distillation only.
• Evaluate using Dice, NSD, and HD95 to assess volumetric and boundary fidelity, and analyze CPU/GPU inference performance across hardware.

Experimental results

Research questions

• RQ1 Can logit-based knowledge distillation enable aggressive compression of a high-performance segmentation model without changing the deployment pipeline?
• RQ2 Does a distilled, compact nnU-Net maintain clinically meaningful segmentation accuracy under fixed on-premises constraints across MRI and CT modalities?
• RQ3 To what extent does channel reduction combined with KD improve inference efficiency on CPU and GPU hardware in real-world clinical workflows?
• RQ4 Is the KD-based deployment framework generalizable across imaging modalities without task-specific redesign?

Key findings

• Distilled, compact models preserve nearly all teacher performance at 1/2 scale ( Dice 79.48 vs 81.65; 99.6% relative fidelity to the teacher); at 1/4 scale KD recovers a substantial portion of lost performance (60.8% of degraded accuracy).
• Knowledge distillation stabilizes aggressively compressed models, reducing boundary degradation (HD95) and maintaining anatomical coherence under compression.
• Channel reduction yields substantial efficiency gains with over 36% memory and up to 67% CPU latency reduction, while maintaining inference compatibility with existing workflows.
• Across modalities (brain MRI to abdominal CT BTCV), the distillation framework generalizes without architecture changes, with distilled models outperforming non-distilled baselines under the same computational budgets.