[论文解读] EvalBlocks: A Modular Pipeline for Rapidly Evaluating Foundation Models in Medical Imaging

Jan Tagscherer, Sarah de Boer+8 more|arXiv (Cornell University)|Jan 7, 2026

Radiomics and Machine Learning in Medical Imaging被引用 0

一句话总结

EvalBlocks 提供基于 Snakemake 的模块化工作流，能够在医疗影像任务中快速评估多种基础模型，具备缓存、可重复性和可在集群环境中执行的能力。

ABSTRACT

Developing foundation models in medical imaging requires continuous monitoring of downstream performance. Researchers are burdened with tracking numerous experiments, design choices, and their effects on performance, often relying on ad-hoc, manual workflows that are inherently slow and error-prone. We introduce EvalBlocks, a modular, plug-and-play framework for efficient evaluation of foundation models during development. Built on Snakemake, EvalBlocks supports seamless integration of new datasets, foundation models, aggregation methods, and evaluation strategies. All experiments and results are tracked centrally and are reproducible with a single command, while efficient caching and parallel execution enable scalable use on shared compute infrastructure. Demonstrated on five state-of-the-art foundation models and three medical imaging classification tasks, EvalBlocks streamlines model evaluation, enabling researchers to iterate faster and focus on model innovation rather than evaluation logistics. The framework is released as open source software at https://github.com/DIAGNijmegen/eval-blocks.

研究动机与目标

• 在医疗影像模型开发过程中，说明快速、可重复评估基础模型的必要性。
• 提出一个基于 Snakemake 的模块化、即插即用的评估框架。
• 通过在若干医疗影像分类任务上评估多种基础模型来演示框架。
• 强调集中实验跟踪与缓存如何加速迭代并降低后勤负担。

提出的方法

• 实现一个模块化管道，其中区块执行特征提取、可选聚合和评估，作为 Snakemake 规则。
• 支持无缝集成新的数据集、基础模型、聚合方法和评估策略。
• 使用集中配置来声明实验，并在单命令下实现可重复结果。
• 实现面向生产的缓存和适合集群环境（如 Slurm）的并行执行。
• 在三个恶性肿瘤分类任务上评估五种基础模型，使用多种聚合和评估策略。
• 提供嵌入分析（LDA、PCA、t-SNE）的可视化区块，以帮助解释。

Figure 1 : In our framework, pipeline steps are implemented as self-contained blocks. Foundation models embed input patches, and these feature embeddings can be optionally aggregated and then evaluated. The pipeline blocks can be freely extended and plugged into each other, enabling fast, reproducib

实验结果

研究问题

• RQ1 模块化、可重复使用的管道如何加速医疗影像中基础模型的迭代评估？
• RQ2 不同聚合策略对跨模型与模态的下游分类性能有何影响？
• RQ3 集中实验跟踪与缓存是否能在比较多种基础模型时缩短评估时间并提高可重复性？
• RQ4 各种评估策略（kNN、线性探针、可视化）在区分不同数据集的恶性肿瘤方面有何比较？
• RQ5 在本地和集群基础设施上运行此框架以用于内部数据集是否具备可行性？

主要发现

• EvalBlocks 实现了跨多模型、任务与配置的自动化、可重复评估。
• 该框架通过缓存与并行执行支持快速迭代，缩短实际耗时。
• 不同模型在数据集与模态上的优势各异，在演示中没有单一模型主导所有任务。
• 聚合方法（如跨模态取均值）可能影响性能并促进融合策略的快速原型设计。
• 可视化分析为嵌入分离性提供可解释的见解，并指引后续模型开发。

Figure 2 : A visualization of model results on our CT datasets created by running EvalBlocks, with error bars depicting the standard deviation across folds. While CT-FM [ 9 ] and Curia [ 3 ] perform best on PANORAMA [ 1 ] , UMedPT [ 11 ] is slightly more accurate on AMARA. Our pipeline allows for fa